                                                              2225

 1    UNITED STATES DISTRICT COURT
      EASTERN DISTRICT OF NEW YORK
 2    ------------------------------x
      THE CITY OF NEW YORK, et al.,
 3    
                         Plaintiffs,          88 Civ. 3474 (JMcL)
 4    
                 v.                           
 5    
      UNITED STATES DEPARTMENT OF COMMERCE,
 6    et al., 
      
 7                       Defendants.
      ---------------------------------x
 8    CITY OF ATLANTA, et al.,
                     
 9                       Plaintiffs,
      
10                v.                          92 Civ. 1566 (JMcL)
                                              
11    MOSBACHER, et al.,
      
12                       Defendants.
      ---------------------------------x
13    FLORIDA HOUSE OF REPRESENTATIVES, 
      et al.
14    
                         Plaintiffs,
15                v.                          92 Civ. 2037 (JMcL)
      
16    BARBARA H. FRANKLIN, Secretary of
      Commerce,
17    
                         Defendant.
18    --------------------------------x
      
      
19                                            May 27, 1992
                                              9:30 a.m.
20    
      
21    
      Before:
22    
                 HON. JOSEPH M. McLAUGHLIN,
23    
                                              Circuit Judge
24    

25    
                                                              2226

 1               MR. GOLDIN:  Your Honor, before we begin, while 

 2    the witness is taking the stand, if I could raise a small 

 3    housekeeping matter.  

 4               THE COURT:  Yes.  
                                   

 5               MR. GOLDIN:  We had two flip charts that I 

 6    sketched yesterday which I would like to mark as 

 7    illustrative exhibits to accompany the testimony.  Mr. Baron 

 8    has informed me that he has no objection.  

 9               THE COURT:  You will arrange for its 

10    reproduction?  

11               MR. GOLDIN:  We will do that.  We will consult 

12    with the Court and determine how that is customarily done.  

13               THE COURT:  I haven't any idea, and you needn't 

14    consult.  Do it.  

15               MR. GOLDIN:  Thank you, your Honor.  

16    CROSS-EXAMINATION (continued) 

17    BY MR. GOLDIN:

18         Q.    Dr. Wachter, good morning.

19         A.    Good morning, Mr. Goldin.

20         Q.    In your direct examination yesterday you 

21    testified that you had not expected the Bureau or the 

22    Secretary to employ loss function analyses in connection 

23    with the question of whether to adjust the 1990 census, is 

24    that right?

25         A.    For formal decision-making purposes, for formal 
                                                              2227

 1    purposes, that's right.

 2         Q.    Indeed, it was your testimony that you thought it 

 3    would be inappropriate to use loss function analyses in 

 4    connection with the decision, is that right?

 5         A.    That is not precisely right.  There are two ways 

 6    of using loss function analyses.  One is for formal decision 

 7    making, the kind of use that Dr. Fisher described.  The 

 8    other is to shed light in an exploratory way on the relative 

 9    accuracy of two sets of estimates.  I favored the latter, 

10    but I considered the former to be inappropriate.

11         Q.    You believed that it was appropriate to use loss 

12    function analyses to shed light on the relative accuracy of 

13    comparative statistics?

14         A.    I did, and I recommended that in my 

15    recommendation to the Secretary.

16         Q.    Let me show you a document which has been marked 

17    for identification as Plaintiffs' Exhibit 741.  

18               MR. GOLDIN:  May I ask the Court's indulgence a 

19    moment, your Honor?  

20               THE COURT:  Yes.  

21               (Pause) 

22               MR. GOLDIN:  Might I approach the witness, your 

23    Honor?  

24               THE COURT:  Sure.

25         Q.    Is this a document that you have seen previously, 
                                                              2228

 1    Dr. Wachter?

 2         A.    It looks like a document you shared with me at 

 3    the deposition.

 4         Q.    This is a printout of an electronic mail 

 5    communication you transmitted to other members of the 

 6    Special Advisory Panel on or about November 13, 1989, is 

 7    that right?

 8         A.    I believe so, yes.

 9         Q.    In this document you comment generally on 

10    Department of Commerce's draft adjustment guidelines and 

11    specifically on a letter by Kirk Wolter discussing those 

12    draft guidelines, is that right?

13         A.    Yes.  I was suggesting language for a letter that 

14    the Special Advisory Panel eventually did draft.  

15               MR. GOLDIN:  I offer Plaintiffs' Exhibit 741, 

16    your Honor.  

17               MR. BARON:  No objection.  

18               THE COURT:  741 is admitted. 

19               (Plaintiffs' Exhibit 741 for identification was 

20    received in evidence).

21         Q.    Dr. Wachter, I call your attention to page 3 of 

22    this letter.  Would you turn to that, please.  Would you 

23    look in the middle of that page at the heading "Guideline 

24    6."  Do you see that?

25         A.    I do.
                                                              2229

 1         Q.    The second paragraph thereunder says, "Kirk 

 2    suggests that it would be useful to have some explicit loss 

 3    functions mentioned at least as examples.  It would 

 4    certainly make our task as advisers easier and more direct."  

 5               Do you see that?

 6         A.    I do.

 7         Q.    What you had in mind when you wrote that were 

 8    loss functions of the kind presented in the Wolter/Causey 

 9    paper in the National Academy panel report and the like, is 

10    that correct?

11         A.    That's correct. 

12         Q.    Dr. Wachter, you would agree, would you not, that 

13    the PES provides us with the principal evidence we have 

14    about undercount in the 1990 census?

15         A.    It is supplemented by demographic analysis, but 

16    it is the principal evidence we have, yes.

17         Q.    You certainly agree that it is the principal 

18    evidence that we have about census undercount at subnational 

19    levels, is that right?

20         A.    That's right.

21         Q.    During your direct examination yesterday you 

22    testified, did you not, that there are uncertainty intervals 

23    surrounding the demographic analysis derived estimate of 

24    differential undercount in the 1990 census?

25         A.    Yes.
                                                              2230

 1         Q.    You further testified, did you not, that because 

 2    of that uncertainty, you could not be sure that differential 

 3    undercount in the 1990 census is more severe than in 

 4    previous censuses for which demographic analysis estimates 

 5    are available, is that right?

 6         A.    I was testifying about a particular differential, 

 7    the differential between blacks and nonblacks.  I did 

 8    testify than we could not be sure, looking at the historical 

 9    series, whether that differential was lower or higher than 

10    the differentials in the last 50 years.

11         Q.    There were uncertainty intervals for the prior 

12    demographic analysis estimates of differential undercount, 

13    too, isn't that right?

14         A.    The Bureau has retrospectively computed updated 

15    uncertainty intervals for the demographic analysis estimates 

16    for 1980.  It may be that they have retrospectively computed 

17    them back before 1980.  If they have, those numbers are not 

18    given in the D-11 report, and I have not seen them.  

19    However, there is uncertainty in all those estimates.

20         Q.    As a professional demographer, you do regard 

21    those prior estimates as having uncertainties, is that 

22    right?

23         A.    Certainly.

24         Q.    Have you attempt to do calculate the uncertainty 

25    intervals that surround them?
                                                              2231

 1         A.    For the 1950, '60, and '70 figures, no, I 

 2    haven't.

 3         Q.    Do you have any reason to believe that those 

 4    uncertainty intervals are narrower than those surrounding 

 5    the 1990 demographic analysis estimate of differential 

 6    undercount?

 7         A.    In my best judgment, they are likely to be wider, 

 8    but there is a reason that they could be narrower.  One of 

 9    the principal sources of uncertainty in 1980 and 1990 is our 

10    uncertainty about undocumented immigrants, and undocumented 

11    immigrants were not so large a source of uncertainty in 1960 

12    and before.  So it is something of an open question, but my 

13    best expectations would be that the intervals would be wider 

14    rather than narrower for the earlier years.

15         Q.    In fact, undocumenteds were a greater source or 

16    at least as great a source of uncertainty in 1970 than in 

17    1980, isn't that right?

18         A.    The numbers at stake are smaller, but I think the 

19    uncertainties are probably greater.

20         Q.    Greater for the previous years?

21         A.    Are greater in 1907, yes, I think that is 

22    probably so.

23         Q.    The effect of the amnesty was to reduce the 

24    uncertainty somewhat, is that right?

25         A.    It reduced the uncertainty in one component and 
                                                              2232

 1    it enhanced the uncertainty in another component.  The 

 2    amnesty included a provision for special agricultural 

 3    workers, which introduced a new wild card.  But the amnesty 

 4    reduced some of the uncertainty.

 5         Q.    Dr. Wachter, during your service as a member of 

 6    the Special Advisory Panel, was there at one point a 

 7    proposal considered by the Bureau in connection with the 

 8    issue of adjusting the 1990 census that was known as Combo?

 9         A.    That's right, yes.

10         Q.    What was that?

11         A.    In brief, the notion was to use the demographic 

12    analysis estimates to readjust the adjusted estimates so 

13    that the adjusted counts would agree with the demographic 

14    analysis totals at the national level, to use multipliers 

15    after the whole adjustment was computed.  So that would be a 

16    combination of demographic analysis with the PES.

17         Q.    Did you have any feelings yourself about the 

18    wisdom of going ahead with Combo?

19         A.    I believed it would have been unwise to go ahead 

20    with Combo, and I stated that both in testimony before the 

21    House subcommittee and I believe in a letter to the Bureau.

22         Q.    So you made your position on Combo known as 

23    strongly as you could?

24         A.    I made it known strongly.  I think it was a 

25    fairly bad idea on the face of it.  If it had had more 
                                                              2233

 1    likelihood of being adopted, I would have made my position 

 2    known even more strongly.

 3         Q.    In your Congressional testimony opposing Combo, 

 4    did you portray yourself, in effect, as trying to stop a 

 5    snowball that was rolling downhill, a project that was 

 6    rapidly gathering steam that you thought had to be derailed?  

 7               THE COURT:  Or gathering moss?  

 8               MR. GOLDIN:  That kind of thing.

 9         A.    I don't remember how I would characterize the 

10    testimony.  But I certainly wanted to stop Combo, and I 

11    indeed tried to enlist Mr. Rifkind's enthusiasm for stopping 

12    Combo.  I wanted to give the adjustment the best chance it 

13    could, and I felt that Combo would introduce large 

14    additional uncertainties due to the large uncertainties in 

15    the demographic analysis, and it would essentially spoil 

16    whatever quality the adjusted counts had.  That didn't seem 

17    to me a fair thing to do.  We wanted to give the adjusted 

18    counts the best chance we do.

19         Q.    You did, in fact, stop Combo or Combo in any 

20    event got stopped, is that correct?

21         A.    I don't think I was the only person who 

22    recognized that this was a bad idea, but I hope I had some 

23    role in stopping it.  I wished to --

24         Q.    It did get stopped, right?

25         A.    It was stopped.  It was stopped.
                                                              2234

 1         Q.    When, Dr. Wachter, did you become aware that the 

 2    Bureau was planning to smooth variances for the production 

 3    of smoothed PES adjustment factors?

 4         A.    I don't know that with precision.  I know I 

 5    became very much aware of it during a conference telephone 

 6    call among Bureau staff and special advisory panels in which 

 7    this issue surfaced as a major issue.

 8         Q.    Did you make known your opposition to variance 

 9    smoothing?  

10               MR. BARON:  Your Honor, I will object that this 

11    is beyond the scope of the direct testimony.  

12               THE COURT:  I will permit it.

13         A.    There was an open scientific discussion, and I 

14    raised in the course of that discussion several problems 

15    which I saw with variance smoothing, what is now called 

16    variance presmoothing, several potential problems, several 

17    undesirable effects that I thought the particular 

18    presmoothing had.  

19               I wouldn't say that I was taking some kind of 

20    established position.  I was, as Dr. Tukey was and as other 

21    participants in the conference call were, we were vetting or 

22    airing some of the pros and cons and suggesting some 

23    alternatives.

24         Q.    Did you write letters to the Bureau opposing 

25    variance smoothing?
                                                              2235

 1         A.    We are talking about a very late stage now, at 

 2    the end of May, fairly shortly before the reports were 

 3    filed.  I don't believe I wrote any letters.  I made some of 

 4    my worries clear in the phone call, and I went on then to 

 5    analyze the data on the issue and develop more firm 

 6    opinions, which were then expressed in my recommendation.

 7         Q.    Did you apprise Congress of your concerns about 

 8    variance smoothing?

 9         A.    What period?  What dates are you referring to.

10         Q.    At any time before the submission of your 

11    recommendation.

12         A.    By my memory, I don't believe there was a 

13    Congressional hearing between the time when variance 

14    presmoothing surfaced and I submitted my recommendation.  

15    But there was a hearing following the submission of my 

16    recommendation at which the issues of variance presmoothing 

17    were given a large amount of attention both by Dr. Tukey and 

18    myself.

19         Q.    Before the submission of your recommendation, you 

20    could have written a letter to any of a number of 

21    representatives in Congress, count? 

22         A.    To what purpose?

23         Q.    To point out your opposition to variance 

24    smoothing.

25         A.    Congress hadn't solicited any kind of input from 
                                                              2236

 1    me.  They solicited my views on particular questions which 

 2    were incorporated in invitations to me to testify.  When I 

 3    testified to Congress, I addressed the issues that Chairman 

 4    Sawyer or Chairman Coale had asked me to address.  I wasn't 

 5    somehow using Congress, trying to use Congress as a platform 

 6    for my views.  I was communicating my recommendations to the 

 7    Secretary and preparing the best recommendations I could.

 8         Q.    Did you generally make every effort you could to 

 9    stop variance smoothing?

10         A.    No, I wasn't trying to stop it.  I was airing the 

11    problems that I saw which had not been fully assessed.  

12    Until they were fully assessed, it was an open question in 

13    my mind what the final weight of evidence would be.  But I 

14    saw some real problems in it, and later examinations of the 

15    data have borne those problems out.  

16               Unlike Combo, it wasn't an issue that somehow on 

17    the face of it was necessarily foolish.  It turned out 

18    perhaps to be foolish, but the situation with Combo and the 

19    situation with variance presmoothing, one is a less 

20    technical issue than the other.

21         Q.    In your recommendation to the Secretary, 

22    Defendants' Exhibit 39, you spend a fair bit of text, 

23    specifically the eight pages from page 33 to page 41, 

24    writing about smoothing, don't you?

25         A.    I do, yes.
                                                              2237

 1         Q.    To prepare that text and to make your 

 2    recommendation in general, you spent a good bit of time 

 3    reviewing the Bureau's approach to smoothing, is that 

 4    correct?

 5         A.    I was aided in that review by two sources.  One 

 6    was a consultant working for me, Christopher Genovese, who 

 7    reviewed the numbers under my supervision.  The other was 

 8    that I paid as much attention as I could, a great deal of 

 9    attention, to what Dr. Tukey had to say and to what Dr. 

10    Hoaglin had to say.  Dr. Hoaglin prepared a report which was 

11    included in the joint Ericksen/Tukey/Estrada and Wolter 

12    report.  And he shared some of those results with us, and I 

13    studied those results.  

14               So I had devoted attention to the issue.  But the 

15    issue, as I say in my report, is an extraordinarily complex 

16    one and many aspects of it have come to light since I wrote 

17    my report.

18         Q.    In fact, you coined the term "presmoothing" for 

19    variance smoothing, isn't that right?

20         A.    I did, yes.

21         Q.    So you would say smoothing is a subject with 

22    which you have some familiarity?

23         A.    This kind of smoothing is an area of multivariate 

24    analysis, and multivariate analysis is one of my 

25    specialties.  There is another area of smoothing, time 
                                                              2238

 1    series smoothing, which is not one of my special areas.

 2         Q.    I won't get into that one.  I will ask you to 

 3    take a look, Dr. Wachter, at two documents, one of which is 

 4    Defendants' Exhibit 68 and the other one of which is a copy 

 5    of a page from Defendants' Exhibit 66.  

 6               MR. GOLDIN:  May I approach the witness, your 

 7    Honor?  

 8               THE COURT:  Yes.  

 9               MR. BARON:  Your Honor, I wish to renew my 

10    objection.  We have had testimony about this page, as you 

11    know, from several witnesses in this case, but one of the 

12    witnesses was not Professor Wachter.  I did not ask him any 

13    questions about this exhibit and he did not testify about 

14    the subjects of smoothing or presmoothing.  

15               MR. GOLDIN:  May I respond, your Honor?  

16               THE COURT:  Yes.  

17               MR. GOLDIN:  We had extensive testimony from Dr. 

18    Ericksen, who did not testify about smoothing or 

19    presmoothing.  

20               MR. BARON:  That is incorrect.  

21               THE COURT:  Go ahead.  

22               MR. GOLDIN:  And defendants introduced 

23    Defendants' Exhibits 68 and this page from 66 in the form of 

24    a blow-up chart for purposes' of cross-examining Dr. 

25    Ericksen.  They have used these exhibits to cross-examine, I 
                                                              2239

 1    believe, the majority of plaintiffs' witnesses.  

 2               THE COURT:  How long will it take you to go 

 3    through this witness with this?  

 4               MR. GOLDIN:  I only have a few questions on this, 

 5    your Honor.  It should be with three minutes.  

 6               THE COURT:  All right.  I will overrule the 

 7    objection.  

 8               MR. GOLDIN:  Thank you, your Honor.  

 9               THE COURT:  And start my clock.  

10               MR. GOLDIN:  Three minutes for the questions.  I 

11    wasn't counting the answers.

12         Q.    Dr. Wachter, Defendants' Exhibit 66, the page 

13    extract that we have provided you with, describes the 

14    Bureau's PES production smoothing model, is that correct?

15         A.    It is not labeled 66.  

16               THE COURT:  Tell us what it is and we will 

17    believe you.

18         A.    Yes, looks like it, the one with sigma squared I 

19    on it.

20         Q.    Right.  Defendants' Exhibit 68 -- 

21               THE COURT:  Is that a fraternity?  

22               THE WITNESS:  It is becoming one, yes.

23         Q.    Defendants' Exhibit 68, and in particular the 

24    circled matter three lines up from the bottom of the page -- 

25               THE COURT:  That looks like my circle.  
                                                              2240

 1               MR. GOLDIN:  Could be.  

 2               THE COURT:  How did you get a xerox copy of it?  

 3    Well, anyway.

 4         Q.    -- involves modifying one term and one formula 

 5    there suggesting that the covariance matrix V sub 3 replace 

 6    sigma squared I as the covariance matrix for little v, which 

 7    is the residual nonsampling nonlinearity, is that right?

 8         A.    Right.

 9         Q.    Defendants' Exhibit 68 thus embodies a suggested 

10    alternative proposed by defendants' expert David Freedman, 

11    is that right?

12         A.    That's right.

13         Q.    In your recommendation, Defendants' Exhibit 39, 

14    to the Secretary, you make no mention of any concerns about 

15    the Bureau's use of sigma squared I as the covariance matrix 

16    for little v, is that right?

17         A.    That's right.

18         Q.    You make no mention there of V sub 3 or any other 

19    covariance matrix for V sub 3, is that right?

20         A.    That's right.

21         Q.    That is because when you wrote your 

22    recommendation you thought sigma squared I was a reasonable 

23    choice for the covariance matrix for little v, is that 

24    right?

25         A.    No.
                                                              2241

 1         Q.    You did not think it was a reasonable choice?

 2         A.    I recognized in my report that the smoothing had 

 3    turned out to be problematic.  There was prima facie 

 4    evidence which I gave in my report that the smoothing was 

 5    doing strange things.  But the data on the smoothed 

 6    adjustment factors came only a couple of weeks before the 

 7    report, and issues like the second equation required a good 

 8    deal of study, and that study came after my report.  

 9               It was later that I began to trace some of the 

10    problems in the smoothing to the equation you see.  I was 

11    actually stimulated to think in this direction not by 

12    Professor Freedman but by Dr. Tukey, who discussed this 

13    issue with me and proposed, I believe in a memo he sent 

14    around, an alternative that in several respects was similar 

15    to this one.  So I was actually thinking about this in the 

16    lines Dr. Tukey was suggesting before Dr. Freedman and I 

17    began to discuss the question.

18         Q.    This was not an issue that was in your mind when 

19    you wrote your recommendation, is that right?

20         A.    That's right.  I hadn't traced the problems down 

21    to this kind of source.

22         Q.    You think that V sub 3 is a reasonable 

23    alternative to sigma squared I here?

24         A.    Yes, I do.  I do.  There are other reasonable 

25    alternatives, but this is one that should certainly be 
                                                              2242

 1    looked at.

 2         Q.    Had you looked at it yourself?

 3         A.    I have not conducted a full analysis of it 

 4    myself.  I have looked at similar alternatives in some 

 5    stylized cases.  But my testimony was not directed at 

 6    smoothing, and I left off a full examination of these 

 7    issues.

 8         Q.    Dr. Wachter, in your account of the CAT 

 9    simulations contained in your recommendation to the 

10    Secretary, you report on numbers of districts improved and 

11    worsened and on proportions of districts improved and 

12    worsened by adjustment, but you do not say by how much 

13    districts are improved or worsened, is that right?

14         A.    That's right.

15         Q.    In your report you do not differentiate for any 

16    given simulation between a district that is improved a great 

17    deal and one that is improved slightly, that is right?

18         A.    That's right.

19         Q.    When you count districts in your report, you are 

20    simply counting districts improved, even if greatly 

21    improved, as trading off evenly with districts made worse, 

22    even if only made only slightly worse, is that right?

23         A.    I had looked at the degrees of improvement 

24    myself, but in my report I don't report that.  I found that 

25    the improvements or lack of improvements tended to be fairly 
                                                              2243

 1    evenly spread rather than extreme at these local levels.

 2         Q.    But you don't report those values?

 3         A.    I didn't report that.

 4         Q.    This approach that you used in the recommendation 

 5    to measure improvement or worsening is what you called unit 

 6    counting, is that right?

 7         A.    Yes, it can be called unit counting.

 8         Q.    That is a kind of loss function, isn't it?

 9         A.    I don't think of it as a loss function, but it 

10    can be used as a loss function.  It can be expressed as a 

11    loss function.

12         Q.    Do you recall at your deposition that you 

13    described it to me as being an example of a loss function?

14         A.    I recall that in a general way.

15         Q.    Do you recall it specifically?

16         A.    As I just said, it can be viewed as a loss 

17    function.

18         Q.    Let me call your attention to pages 349 to 50 of 

19    your deposition.  

20               MR. GOLDIN:  May I approach the witness, your 

21    Honor?  

22               THE COURT:  Yes.

23         Q.    If you would look towards the bottom of page 349, 

24    Dr. Wachter, do you see the following exchange?

25              "Q.    When you refer to unit counting in your 
                                                              2244

 1    SIMCAT, are you talking about counting the number of units 

 2    improved by adjustment and the number of units not improved 

 3    by adjustment?

 4              "A.    Yes, with the fraction of all units 

 5    improved or not improved.  That is one example of a loss 

 6    function.

 7              "Q.    In that loss function, what is not taken 

 8    into consideration is the amount by which a unit is improved 

 9    or worsened, is that correct?

10              "A.    In that loss function, it is not."

11               Does that refresh your recollection that you 

12    described unit counting as a loss function at your 

13    deposition?

14         A.    Yes.

15         Q.    That is the only loss function or measure of 

16    closeness that you used in your recommendation to the 

17    Secretary, is that right?

18         A.    I present the results in graphical form and not 

19    as results of a loss function, but that is the only -- I 

20    think it is fair to say that is the nearest thing to a loss 

21    function that I use in my recommendation.

22         Q.    You regard your use of that loss function as a 

23    personal and political choice on your part, is that right?

24         A.    Yes.

25         Q.    When I took your deposition about two months ago, 
                                                              2245

 1    you told me, did you not, that you were investigating the 

 2    application of other loss functions or measures of closeness 

 3    to the CAT simulations?

 4         A.    Yes.

 5         Q.    In fact, did you not tell me at your deposition 

 6    that you expected to have in hand by the time of trial 

 7    results of different measures of closeness or loss functions 

 8    to the CAT simulations?

 9         A.    I did expect to have them in hand.  I have 

10    extensive printouts of such results, but in the time period, 

11    the number of reports that I was able to complete or the 

12    most that I was able to complete, that report is not one 

13    that I was able to complete.  I hoped to complete it, but at 

14    the end, as May 1st came, I didn't have the time to 

15    completely write that up.

16         Q.    In the report on CAT simulations that you 

17    prepared for this litigation and the defendants have 

18    submitted to the Court, Defendants' Exhibit 46, the only 

19    measure of closeness or loss function that is used is unit 

20    counting again, is that right?

21         A.    That is what is in this report.

22         Q.    Let's take an example.  If you would turn again, 

23    please, to the page following page 22 of the appendix in 

24    Defendants' 46 and look again at figures 9 and 10 on that 

25    page.  I just want to look at I Figure 9 for purposes of 
                                                              2246

 1    illustrating how unit counting works.  Do you have that 

 2    figure before you, Dr. Wachter?

 3         A.    Yes.

 4         Q.    Figure 9 tells us about 250 CAT simulations for 

 5    your New York site using fit 1, is that right?

 6         A.    Right.

 7         Q.    In terms of numveric accuracy, Figure 9 tells us 

 8    that adjustment produced some improvement for the majority 

 9    of districts, 249 out of 250 simulations, is that right? 

10         A.    Yes.

11         Q.    Still in terms of numeric accuracy, Figure 9 

12    tells us that on average over those 250 simulations, 73 

13    percent of districts were improved, right?

14         A.    In numeric terms, yes.

15         Q.    In distributive terms, Figure 9 tells us that 

16    adjustment produced some improvement for the majority of 

17    districts 204 out of 250 simulations, is that right?

18         A.    Yes.

19         Q.    Also in distributive terms, Figure 9 tells us 

20    that on average over those 250 simulations, 58 percent of 

21    districts were improved, is that right?

22         A.    Right.  These are the figures in the absence of 

23    systematic heterogeneity.

24         Q.    Figure 9 does not tell us by how much a district 

25    was improved or made worse, is that right?
                                                              2247

 1         A.    That's right.

 2         Q.    I want to turn to an illustration of the unit do 

 3    you not in loss function you use in your recommendation and 

 4    in this report and which we also discussed at your 

 5    deposition.  

 6               MR. GOLDIN:  With the Court's indulgence, again, 

 7    I would like to sketch this hypothetical briefly on a flip 

 8    chart.  

 9               THE COURT:  All right.

10         Q.    Dr. Wachter, I would like you to assume that we 

11    have five blocks -- A, B, C, D, and E -- and we have three 

12    counts, the census, a proposed adjustment, and what we know 

13    in this hypothetical to be the truth.  In the census count 

14    the population is found for the five blocks are two persons 

15    in block A, one person in each of blocks B, C, and D, zero 

16    persons in block E.  

17               A proposed adjustment would leave the population 

18    of block A unchanged at two people.  It would add one person 

19    to each of the four remaining blocks so that the populations 

20    of blocks B, C, and D would become two persons in each 

21    block, and the population of block E would become one 

22    person.  

23               The truth is, and it is given to us to know in 

24    this hypothetical, that there are two people in each block.  

25               Using a unit counting loss function from the 
                                                              2248

 1    standpoint of distributive accuracy, you would say, would 

 2    you not, that the census count achieves greater distributive 

 3    accuracy?  Is that right?  Can you see the hypothetical on 

 4    the chart?

 5         A.    Yes.  

 6               This is a case where the true undercount is 50 

 7    percent.  In this case the census achieves better 

 8    distributive accuracy in terms of unit counting and the 

 9    adjustment achieves greater numeric accuracy in terms of 

10    unit counting.

11         Q.    I would like you to make it clear that this 

12    example does not depend upon small numbers to replicate it.  

13    If you will bear with me, with slightly larger numbers.  

14    Although not those, which bear upon a later example.  

15               Let us now envision a United States of five 

16    states, A, B, C, D, and E, which in the census are counted 

17    with populations as follows:  

18               State A has 7 million people.  States B, C, and D 

19    have 5 million people each.  State E has 3 million people.  

20    An adjustment again is proposed under which each of states 

21    A, B, C, and D would have 6 million people, state E would 

22    have 5,999,999 people.  You will not be surprised to know 

23    that the truth is that each of these states has 6 million 

24    people.  

25               I take it, Dr. Wachter, that, again using a unit 
                                                              2249

 1    counting approach, looking at distributive accuracy, you 

 2    would conclude that the census count here is more accurate, 

 3    is that correct?  I have a calculator if you need it.

 4         A.    25 altogether, so we have 20 percent in B, C, and 

 5    D.  So the distributive accuracy of the shares of B, C, and 

 6    D are correct in the census, and the shares of A and E are 

 7    incorrect.  So unit counting gives that result.  This is 

 8    another example of the kind you gave me with smaller 

 9    numbers.  

10               (Continued on next page) 

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2250

 1         Q.    And once again, unit counting tells you in this 

 2    example that the census count is more accurate from a 

 3    distributive point of view, is that right?

 4         A.    That's right.  

 5               MR. GOLDIN:  Your Honor, I would like to mark the 

 6    two charts we just created as illustrative examples.  

 7               THE COURT:  Done.  

 8               MR. GOLDIN:  Thank you. 
                                         

 9    BY MR. GOLDIN:

10         Q.    Dr. Wachter, a little while ago yesterday, we 

11    were looking at your piece on the homogeneity assumption in 

12    the 1990 census adjustment and introduction.

13               Would you go back to that again for a moment and 

14    I will try to identify which exhibit it is.  

15               MR. BARON:  Exhibit 41.  

16               MR. GOLDIN:  Thank you.

17         Q.    Would you look at the second sentence of that 

18    exhibit on the first page.

19               Do you have that before you? 

20               Do you have that, Dr. Wachter?

21         A.    I do, yes.

22         Q.    The second sentence there, you are, in part, 

23    quoting the Secretary to the effect that the homogeneity 

24    assumption for census omission rates is the assumption that 

25    quote from the Secretary the probability of being missed by 
                                                              2251

 1    the census is constant for each person within an age, race, 

 2    Hispanic origin, sex and tenure category in a geographical 

 3    area.

 4               Do you see that?

 5         A.    I do.

 6         Q.    The Bureau's post-stratification for the PES 

 7    does, in fact, include tenure, that is, whether one lives in 

 8    owned or rented houses, as a variable, is that right?

 9         A.    In some of the post-strata.

10         Q.    In some of the post-strata.

11               In fact, the Bureau considers tenure to be a very 

12    important variable in post-stratification, is that right?

13         A.    Well, I think that's right.  There are some mixed 

14    results, I think, from the smoothing, but on the whole I 

15    think that's right.

16         Q.    Would you take a look at Defendant's Exhibit 31, 

17    the minutes of the CAPE meeting for April 6, 1992, and in 

18    particular turn to attachment one to that document, page one 

19    of that attachment.

20               The attachment starts two pages into the 

21    document.

22               Do you have it there?

23         A.    The one that begins, "Coverage measurement 

24    research and evaluation adjustment of postcensal estimates 

25    March 16, 1992"?
                                                              2252

 1         Q.    No.  The April 6 minutes, which should be 

 2    Defendant's Exhibit 31, front page of which should say, 

 3    "Minutes -- "

 4         A.    I'm sorry, the next exhibit.

 5         Q.    And three pages in you should find attachment 

 6    one?

 7         A.    Yes.

 8         Q.    Which has a table there.

 9               Do you see that?

10         A.    Yes.

11         Q.    That table shows, does it not, that for 

12    non-Hispanic whites in urban areas of over 250,000 in the 

13    northeast, the undercount estimate for owners is negative 

14    2.13 percent, i.e., an overcount, while for nonowners it's 

15    1.16 percent.

16               Is that right?

17         A.    This is the first time I've looked at the table, 

18    but it appears to me that that's what it shows.

19         Q.    And there are similar differences for other parts 

20    of the country, is that right, differences, that is, between 

21    owners and nonowners?

22         A.    That's right.

23         Q.    And for blacks, the lower portion of the page, in 

24    urban areas in the northeast with populations over 250,000, 

25    the undercount estimate for owners is 1.63 percent while for 
                                                              2253

 1    nonowners it's 8.37 percent; is that right?

 2         A.    Right.

 3         Q.    And that disparity continues, according to these 

 4    estimates, for blacks in other parts of the country as well, 

 5    is that right?

 6         A.    That's right.

 7               I'm not sure exactly what these estimates are, 

 8    but the disparity you are describing continue in the table.

 9         Q.    Let me just make sure I understand about your 

10    familiarity or lack thereof with this document.

11               You have not seen this document previously?

12         A.    No, I've not seen this document previously.

13         Q.    Okay.  I will just be on it another moment.

14               Would you turn the page, please, and look at page 

15    two for the attachment where you will see comparable figures 

16    for nonblack Hispanics.

17               Do you have that?  

18               MR. BARON:  Excuse me, your Honor.  I did not 

19    hear whether he had or had not.  

20               THE COURT:  He had not.  

21               MR. BARON:  Then I would lodge an objection to 

22    continuing questioning on a document he has not seen before.  

23               THE COURT:  Overruled.

24         Q.    Do you have that, Dr. Wachter?

25         A.    Yes.
                                                              2254

 1         Q.    And do you see there that for nonblack Hispanics 

 2    we have the same kind of comparison in urban areas of over 

 3    250,000 in the northeast and undercount rate of 0.67 percent 

 4    or owners and 6.72 percent for nonowners and similar results 

 5    elsewhere around the country?

 6         A.    Right.

 7         Q.    And at the bottom of the page we have a similar 

 8    comparison for Asian and Pacific Islanders to the same 

 9    effect.

10               Do you see that?

11         A.    Right.

12         Q.    That suggests that the owner/nonowner or tenure 

13    comparison is an important distinction in 

14    post-stratification, does it not?

15         A.    It does.  It also suggests to me that these new 

16    figures are showing stronger patterns than the production 

17    PES showed, but I would want to go back and compare these 

18    figures with the production PES figures to see whether these 

19    differences have not grown greater.  It looks to me as if 

20    they have.

21         Q.    And the tenure variable is one that you omitted 

22    entirely from the CAT simulations described in Defendant's 

23    Exhibit 46, is that right?

24         A.    That's right, yes.

25         Q.    There is nothing in this Exhibit 31 to suggest 
                                                              2255

 1    that including tenure as a variable in the simulations would 

 2    not have substantially reduced or eliminated the 

 3    heterogeneity the simulations supposedly found, is that 

 4    right?

 5         A.    Well, you've got to be careful here, because 

 6    these are aggregated numbers over whole regions and the 

 7    post-stratification reduces, groups people into much finer 

 8    categories, so this indicates that tenure seems to be a very 

 9    important variable, but it could be that what's being picked 

10    up by tenure is also being picked up in other dimensions of 

11    the post-stratification that are included in the simulations 

12    so it's not an easy leap to go from these aggregate numbers 

13    to what actually happens within post-strata.

14               I've done a number of breakdowns of local level 

15    data which tend to show with the production PES numbers, 

16    which are apparently not these, the tenure distinction 

17    overlaps with some of the other distinctions so that to some 

18    extent you have an either/or category when you get down to, 

19    down to the actual post-strata.  

20               So these are aggregate numbers and the question 

21    is posed at a post-stratum by post-stratum level.

22         Q.    You don't discuss that issue at all in 

23    Defendant's Exhibit 46, do you?

24         A.    No.

25         Q.    You don't provide any evidence in Defendant's 
                                                              2256

 1    Exhibit 46 that including tenure would not have 

 2    substantially reduced or eliminate the heterogeneity the 

 3    simulations supposedly found, is that right?

 4         A.    Looking at an alternative --

 5         Q.    I am just asking you what is in Exhibit 46.

 6         A.    Looking at an alternative with tenure in it was, 

 7    as I believe I mentioned in my deposition, one of the items 

 8    on my agenda, but the 1st of May came when the 1st of May 

 9    did and that was part of the agenda that I have not yet been 

10    able to complete.

11         Q.    So you didn't get to it, it's not in 46 and it's 

12    not described in 46?  

13               THE COURT:  I have the point.

14         Q.    You know the plaintiffs were given a copy of the 

15    file you used for the CAT simulations for the first time on 

16    Friday just before the weekend, is that right?

17         A.    Well, the data on which the simulations are based 

18    was available to Dr. Ericksen, who was a witness for the 

19    plaintiffs, as early as it was available to me, so 

20    plaintiffs' experts, I guess, chose not to pursue this line 

21    of research, but they had the capability to do it on the 

22    same basis that I had.  Those are just data derived from 

23    files that have been available all along.

24         Q.    After July 15, neither you nor Dr. Ericksen had 

25    access to the complete advisory panel file, is that correct?
                                                              2257

 1         A.    We both returned the individual level data.

 2         Q.    And you ran the CAT simulations on an extract 

 3    that you kept from that file, is that right?

 4         A.    That's right.

 5         Q.    Dr. Ericksen did not keep the same extract, to 

 6    your knowledge, is that right?

 7         A.    I don't know what he kept.  

 8               MR. BARON:  Objection.  

 9               MR. GOLDIN:  To his knowledge.  He testified 

10    whether Dr. Ericksen had access.

11         A.    I don't know what Dr. Ericksen kept.

12         Q.    Dr. Wachter, given your familiarity with these 

13    data, would it surprise you to learn that the explanatory 

14    power of tenure as a variable in connection with regressions 

15    for gross omissions and erroneous enumerations in your files 

16    is on the following order.  

17               MR. GOLDIN:  If I may use the flip chart again, 

18    your Honor.  

19               THE COURT:  Yes, all right.

20               It sounds to me like you are testifying.

21               Would it surprise you to learn the following and 

22    now you are about to lecture him.  

23               MR. GOLDIN:  I am offering a hypothetical.

24               We are not in a position to offer it as 

25    testimony.  
                                                              2258

 1               THE COURT:  Ask him to assume something.  

 2               MR. GOLDIN:  Well, I am asking, your Honor, 

 3    whether given the witness' familiarity with these data it 

 4    would be consistent with his knowledge of them to suppose 

 5    that the explanatory power of tenure on a variable is on the 

 6    following order, and I will use as an illustration the first 

 7    line here.

 8         Q.    For the New York site, a regression equation 

 9    using the other variables, race, sex and age without tenure, 

10    as an R squared of 0.1897, when tenure is put in the R 

11    squared is increased to O.5262.

12               Is that consistent with your sense of what is in 

13    these data?

14         A.    Well, you are not describing the regressions you 

15    have done in enough detail for me to know specifically what 

16    you have done and I never look at a regression just in terms 

17    of the R squares.  Dr. Tukey taught us all never to do that.  

18    So I don't know --  

19               THE COURT:  He didn't teach Mr. Goldin.  

20               MR. GOLDIN:  I'm a slow learner, your Honor.

21         A.    So I don't really know whether I would be 

22    surprised or not.

23               If you represent to me that these are the 

24    numbers, I'll think about the numbers, but --

25         Q.    Let me ask you, Dr. Wachter, you have not done 
                                                              2259

 1    any regressions, for example, to assess the impact of tenure 

 2    as an explanatory variable in these data, is that correct?

 3         A.    At the early stages of my analysis, I had a 

 4    number of regressions done for me putting different 

 5    combinations of variables and interactions into the 

 6    equations.  Most of those regressions were done with the 

 7    1,392 post-strata, not with the block level counts.  But I 

 8    would have to review my files to see whether I did 

 9    regressions close to the regressions you have done or not.

10         Q.    But in Defendant's Exhibit 46, you don't report 

11    any evidence of that kind to support the decision to omit 

12    tenure as a variable, is that correct?

13         A.    I don't report this kind of evidence.

14               Figures like these need to be compared against 

15    other sets of figures.  They are interesting on the face of 

16    it, but they don't lead to any conclusions on the face of 

17    it.

18         Q.    I'm not going to go into any detail about T 

19    statistics, but would it surprise you that T statistics 

20    would show for each site for each kind of census error, 

21    tenure was consistently an important variable and sometimes 

22    the most important?  

23               MR. BARON:  I will object to the form.  

24               THE COURT:  It's not good, but I will let it go. 

25               Go ahead.  Can you handle it?
                                                              2260

 1         A.    I wouldn't be overly surprised.  I would want to 

 2    look at the data.

 3         Q.    If all of that --

 4         A.    That is, I would want to look at your form of 

 5    analysis of the data.  I've looked at the data in many other 

 6    ways.

 7               This whole question really depends on what kinds 

 8    of interactions between race and locale and other things you 

 9    have in the equation.

10         Q.    Okay.

11               Let's go back to the results that you actually 

12    reported in Defendant's Exhibit 46.

13               We have already looked at figure 9, we quickly 

14    looked at figure 10, we touched upon this yesterday.  This 

15    appears following page 22 in Exhibit 46.

16               Figure 10, Dr. Wachter, shows numeric accuracy 

17    for districts in New York improved in 239 out of 250 

18    simulations, is that right?

19         A.    Tell me what figure again?

20         Q.    Figure 10, New York site Fit 3.

21         A.    The bottom of the page we looked at before?

22         Q.    Right.

23         A.    Yes.

24         Q.    From the standpoint of distributive accuracy, 

25    adjustment is an improvement in figure 10, 229 out of 250 
                                                              2261

 1    times, is that right?

 2         A.    239, isn't it?

 3         Q.    229 out of 250 simulations the majority of 

 4    districts show improvement from the standpoint of 

 5    distributive accuracy in figure 10, is that right?

 6         A.    Aren't you talking about the figure with 11 out 

 7    of 250 had less than --

 8         Q.    250 minus 21 on the right is 229, correct?

 9         A.    Oh, I'm sorry, you are on the shares.  You are on 

10    the shares.

11         Q.    Yes, distributive accuracy.

12         A.    Yes, that's right.

13         Q.    Okay.

14               And finally, figure 10 adjustment improves 63 

15    percent of districts on average from the standpoint of 

16    either numeric accuracy or distributive accuracy, is that 

17    right?

18         A.    That's right.

19         Q.    Okay.

20               I don't want to become Tukey's, I probably have 

21    gotten to that point --  

22               THE COURT:  You are way beyond Tukey's.

23         Q.    But we do have this large exhibit that came in en 

24    masse.

25               Let me just ask, Dr. Wachter, that you quickly 
                                                              2262

 1    look at the rest of the figures on the following pages 11 

 2    through 24, figures 11 through 24.

 3               I assume you are familiar with these figures.

 4         A.    Yes.

 5         Q.    Can you confirm whether we are talking about Fit 

 6    1 or Fit 3, whichever site we are talking about from the 

 7    standpoint of numeric accuracy, adjustment on average always 

 8    improves at least 56 percent of districts and 50 percent of 

 9    blocks?  

10               MR. BARON:  Your Honor, I suggest that Professor 

11    Wachter --

12         A.    Looks correct.  

13               MR. BARON:  He can take as much time as he wants 

14    to answer that question.  

15               THE COURT:  All right.  He has answered it.

16         Q.    And from the standpoint of distributive accuracy, 

17    referring to those same pages, adjustment on average always 

18    improves at least 52 percent of districts and as much as 76 

19    percent of districts, is that right?

20         A.    I heard the first part, the minimum 52 percent on 

21    the --

22         Q.    From 52 to 76.

23         A.    52 on the districts and you said the maximum was 

24    76?

25         Q.    Right.  That is the range.
                                                              2263

 1         A.    It looks like.  It looks like it.

 2               Yes.

 3         Q.    In fact, for all of these simulations and all of 

 4    the Fits in figures 9 through 24, whichever site, whether 

 5    it's numeric or distributive accuracy, there is never a 

 6    situation in which adjustment on average worsens more 

 7    districts or blocks than it improves, is that right?

 8         A.    We didn't go through the block level counts.

 9               I think those mean improvement figures for the 

10    districts are all greater than 50 percent, that's right.

11         Q.    And there is never a situation in which 

12    adjustment worsens the majority of districts or blocks in a 

13    majority of simulations, is that right?

14         A.    Did you include blocks in your question or just 

15    asked me about districts?

16         Q.    I included blocks. 

17               I can drop them if it will save time.

18         A.    It comes, it comes close in some places in 

19    blocks, but I don't think it's over the boundary.

20         Q.    So the answer is that that's correct?

21         A.    Yes.

22         Q.    Would you look for a moment at figures 11 and 12.

23               Do you have those before you, Dr. Wachter?

24         A.    Yes.

25         Q.    Figure 11 uses Fit 1 to model homogeneity?
                                                              2264

 1         A.    Right.

 2         Q.    Figure 12 uses Fit 3 to model heterogeneity?

 3         A.    Yes.

 4         Q.    And in figure 11 with homogeneity, you show 59 

 5    percent as the mean percentage of districts with improved 

 6    distributive accuracy from adjustment, is that right?

 7         A.    Right.

 8         Q.    While in figure 12 with heterogeneity, the figure 

 9    is 53 percent, is that right?

10         A.    Right.

11         Q.    So if we take percentage of districts with 

12    improved distributive accuracy as our index, with 

13    heterogeneity presence here, adjustment gives us about 90 

14    percent improvement as it does when using homogeneity, is 

15    that right?

16         A.    Well, what we are saying is that sampling error 

17    already introduces very substantial errors and that 

18    heterogeneity along with this --  this form of heterogeneity 

19    along with sampling error worsens things, but not 

20    dramatically.

21         Q.    The sampling error you are referring to is in the 

22    read in variable that is built into this experiment?

23         A.    Yes.

24         Q.    And if we make the same comparison that we just 

25    made for figures 13 and 14, we find, do we not, that 
                                                              2265

 1    adjustment gives us about 95 percent of the improvement with 

 2    heterogeneity that we could obtain with homogeneity, is that 

 3    right?

 4         A.    That's in this case with shares, whereas in this 

 5    case the heterogeneity makes a very big difference to the 

 6    numeric accuracy to the counts.

 7         Q.    Why don't we just focus on distributive accuracy 

 8    for the moment.

 9               Figures 15 and 16 finally show that with respect 

10    to distributive accuracy adjustment gives us about 88 

11    percent, close to 90 percent of the improvement under 

12    heterogeneity that we could have obtained under homogeneity, 

13    is that right?

14         A.    Yes, comparing the means.

15         Q.    Your work with CAT simulations in connection with 

16    the issue of census adjustment was first reported in your 

17    recommendations to the Secretary almost a year ago, is that 

18    right?

19         A.    Yes.

20         Q.    Since then you have continued to work with the 

21    CAT simulations?

22         A.    There was a long period of abeyance and then, 

23    then I returned to the, then I returned to the question.

24         Q.    And you have made substantial modifications in 

25    the CAT simulations reported in Defendant's Exhibit 46 from 
                                                              2266

 1    those that you presented to the Secretary, is that right?

 2         A.    I've made a number of technical improvements and 

 3    refinements and I've streamlined the programs.

 4         Q.    And that would include things like applying the 

 5    simulations to three other sites, changing the approach of 

 6    modeling gross omission and erroneous enumeration rates and 

 7    introducing three new Fits, including Fit 1 and Fit 3 with 

 8    which we have just been concerned, is that right?

 9         A.    Yes.

10         Q.    Now, the actual report that constitutes 

11    Defendant's Exhibit 46 is only six pages long, most of 

12    Defendant's Exhibit 46 is an appendix consisting of the 

13    report on CAT experiments prepared by Mark Hansen, is that 

14    right?

15         A.    Prepared and working together with me by Mark 

16    Hansen.

17         Q.    Will you turn to the front page of that report, 

18    Dr. Wachter.

19               The front page of that report says that this 

20    report on CAT experiments was prepared for Kenneth W. 

21    Wachter by Mark H. Hansen.

22               Do you see that?

23         A.    Yes.

24         Q.    Is that statement correct?

25         A.    Well, Mr. Hansen and I worked together, but it 
                                                              2267

 1    was prepared for me by him.

 2         Q.    So the statement is correct?

 3         A.    What implications are you trying to draw from it? 

 4               It's correct as it stands.

 5         Q.    Mr. Hansen is a student at Berkeley?

 6         A.    A student in statistics.

 7         Q.    To your knowledge, Mr. Hansen himself actually 

 8    conducted the work reported in the appendix to your report?

 9         A.    I wrote the original programs, he introduced 

10    changes in the code to my specification, I went over the 

11    code, he ran the programs that produced the outputs, we then 

12    reviewed together the outputs, did various analyses, he 

13    prepared texts, I suggested changes to the texts, we sort of 

14    argued back and forth and discussed back and forth and he 

15    was -- the final language in the report is his with my 

16    inputs to it.

17         Q.    Have you checked his calculations?

18         A.    I've checked a number of calculations that go in 

19    and the original programs are by me and we have compared 

20    calculations with his versions of the program to 

21    calculations with my version of the program in test cases, 

22    so I have reasonable confidence in the calculations.

23         Q.    But you have not checked all the final 

24    calculations contained or reflected in this report, is that 

25    right?
                                                              2268

 1         A.    I have not --  I have done checking of the 

 2    calculations, I have not checked all the final calculations 

 3    in the report.

 4         Q.    Have you replicated all of Mr. Hansen's 

 5    simulations on your own?

 6         A.    I've replicated simulations with his programs as 

 7    well as with my own programs, so I would say I have a 

 8    hands-on involvement with the data and sometimes at night 

 9    he's there and he runs it, sometimes at night I'm there and 

10    I run it, but I haven't duplicated every calculation that he 

11    has done.

12         Q.    For every simulation, is that right?

13         A.    For every simulation.

14         Q.    Mr. Hansen uses logspline density estimation in 

15    presenting his results, does he not?

16         A.    Well, the figures we have been discussing use 

17    these smooth logspline densities, but the more conventional 

18    histograms for all these results are by my insistence in the 

19    back so that you can see the kind of familiar version of it 

20    as well as this advanced statistical way of presenting it.

21               So if you prefer the ordinary histograms, they 

22    are all in there under the figures A listed as A.

23         Q.    Have you used logspline density estimation in 

24    presenting results in any of your published works?

25         A.    I have never used logspline density estimation 
                                                              2269

 1    myself.  That is one of Mr. Hansen's contributions.  It's a 

 2    graphical and visual presentation of data that is given in 

 3    conventional form later in the report.

 4         Q.    Were you in the courtroom on Friday when Peter 

 5    Bounpane of the Census Bureau testified that he had not read 

 6    any of your reports in connection with this litigation?

 7         A.    I, I was in the courtroom and I heard him say 

 8    that.

 9         Q.    Did you also hear him testify that he had no 

10    reason to believe that any members of the Bureau's committee 

11    on adjustment of postcensal estimates had read any of your 

12    reports?

13         A.    I heard that.

14         Q.    Would it surprise you to learn that the minutes 

15    of the committee on adjustment postcensal estimates show no 

16    mention of your CAT simulations or any of your other 

17    reports?

18         A.    No.

19         Q.    Dr. Wachter, let's turn now to another exhibit.  

20               THE COURT:  Do you have much longer to go?  

21               MR. GOLDIN:  I would guess, your Honor, that I 

22    have about another half hour.  

23               THE COURT:  All right.  Let's take a break.

24               We will resume at 11:00 o'clock.  

25               (Recess)
                                                              2270

 1               THE COURT:  Mr. Goldin, whenever you are ready.  

 2               MR. GOLDIN:  Your Honor, thank you.  

 3    BY MR. GOLDIN:

 4         Q.    Dr. Wachter, would you turn now to Defendant's 

 5    Exhibit 44, which is a piece you wrote called Evidence of 

 6    Heterogeneity from Re-post-stratifications.

 7               Do you have that?

 8         A.    Yes.  

 9               MR. GOLDIN:  Your Honor, at this time I would 

10    like to present to the witness and the court and move into 

11    evidence the July 15 memorandum from Henry Woltman to John 

12    Thompson which we have marked as Plaintiff's Exhibit 740 but 

13    which is, in fact, contained in the administrative record 

14    that defendants have marked as Defendant's Exhibit 1.

15               May I approach the witness?  

16               THE COURT:  Yes.

17               (Handing to the witness)

18               (Pause) 

19    BY MR. GOLDIN:

20         Q.    You are familiar with this memorandum from Henry 

21    Woltman, Dr. Wachter?

22         A.    I am.

23         Q.    In the exhibit that you created, Exhibit 44, you 

24    discuss this alternative post-stratification scheme that Mr. 

25    Woltman proposed, is that right?
                                                              2271

 1         A.    That's right.

 2         Q.    And having revised the grouping of states, Mr. 

 3    Woltman then calculates, does he not, estimates of 

 4    undercount based on his alternative post-stratification 

 5    scheme?

 6         A.    He calculates revised adjusted counts and then he 

 7    includes some undercount estimates, yes.

 8         Q.    His results are not smoothed, is that right?

 9         A.    His results are not smoothed.

10         Q.    So they are to be compared as you do in your 

11    exhibit to the raw undercount estimates produced by the 

12    Bureau using the 1,392 post-strata, is that right?

13         A.    That's the better comparison I think.

14         Q.    In Defendant's Exhibit 44, on page 1, last full 

15    paragraph, you say of the Woltman approach, "For example, in 

16    the Bureau's post-strata, people in Maine, New Hampshire and 

17    Vermont are always grouped together with people in 

18    Massachusetts, Connecticut and Rhode Island.  An alternative 

19    is to split each post-strata, including New Englanders, into 

20    two new post-strata, one with a post-stratum members in 

21    Maine, New Hampshire and Vermont and the other with members 

22    in Massachusetts, Connecticut and Rhode Island.  If the 

23    homogeneity assumption were true, this alternative should 

24    make no systematic difference to the adjusted counts." 

25               Do you see that, Dr. Wachter?
                                                              2272

 1         A.    I do.

 2         Q.    The effect of splitting a particular post-stratum 

 3    as suggested in the paragraph I just read is going to be to 

 4    create two new post-strata, one for southern New England, 

 5    one for northern New England, is that right?

 6         A.    Right.

 7         Q.    And each of those is going to be more or less 

 8    half the size in terms of the sample of the old 

 9    post-stratum, is that right?

10         A.    Right.

11         Q.    And with the reduction in size of the 

12    post-stratum, there is going to be an increase in the size 

13    of the sampling error associated with the estimate, is that 

14    right?

15         A.    That's right.

16         Q.    Now we are dealing with post-strata that are 

17    smaller than the ones actually used in New England in the 

18    production PES and, therefore, have more sampling error 

19    associated with them?

20         A.    That's right.  That's right.

21         Q.    So when you say the paragraph I just said, "If 

22    the homogeneity assumption were true, this alternative 

23    should make no systematic difference to the adjusted 

24    counts," what you mean is that differences that do occur 

25    might be attributable to heterogeneity and might be 
                                                              2273

 1    attributable to nonsystematic, i.e., sampling error, is that 

 2    correct?

 3         A.    Right.

 4         Q.    All right.  Let's look at what actually happens.

 5               At the bottom of page 1 and the carryover 

 6    paragraph you say, "In this example, the alternative does 

 7    make a difference to the adjusted counts.  With the Bureau's 

 8    post-strata, adjustment reduces the share of Maine, New 

 9    Hampshire and Vermont in the U.S. population, the share is 

10    adjusted down by about one percent of its value.  With the 

11    alternative post-strata, adjustment increases the share of 

12    Maine, New Hampshire and Vermont, the share is adjusted up 

13    now also about one percent of its value.  Splitting the 

14    post-strata turns a share loss into a share gain.  This 

15    difference indicates that there is heterogeneity in 

16    undercount rates within the Bureau's post-strata across the 

17    geographical boundary between northern and southern New 

18    England." 

19               Do you see that?

20         A.    Yes.

21         Q.    Now, you say that the difference indicates that 

22    there is heterogeneity, so you are ruling out sampling error 

23    associated with smaller revised state groupings as an 

24    explanation, is that right?

25         A.    No, I'm not ruling that out, but I do discuss 
                                                              2274

 1    those issues in section five of my report.

 2               The answers come out differently so there are 

 3    differences within the post-strata.  Whether those 

 4    differences can be attributed to sampling variability or to 

 5    systematic heterogeneity, to underlying heterogeneity in 

 6    rates is a question that has to be studied in the contect in 

 7    which I study it in the I believe the fifth section of my 

 8    report.

 9         Q.    Let me see if I can understand the paragraph we 

10    were just referring to first.  

11               When you say, "The difference indicates that 

12    there is heterogeneity," are you saying that you have 

13    concluded that the difference is not attributable to 

14    sampling error or are you saying that the difference might 

15    be attributable to sampling error?

16         A.    Well, we don't want to quibble over words.

17               The first paragraph includes the word 

18    "systematic."  If the homogeneity assumption is true, this 

19    alternative should make no difference to adjusted counts.

20               Then we see it makes a difference to the adjusted 

21    counts, that means there must be differences within the 

22    post-strata that have been split, but I don't in this 

23    paragraph --  and that's heterogeneity, but it may not be 

24    systematic heterogeneity, it may be the result of sampling 

25    variability and that's why I go on later in my report to 
                                                              2275

 1    take that question up.

 2               I'm not taking a stand at the top of page 2 about 

 3    whether the heterogeneity is systematic or not.

 4         Q.    So we may be seeing random variability here, is 

 5    that right?

 6         A.    We are seeing a great deal of random variability 

 7    here as we see in the PES, but it's still interesting that 

 8    these regroupings of the states makes such a large 

 9    difference to the actual outcome numbers.

10         Q.    Would you take a look at Henry Woltman's 

11    memorandum, and in particular take a look at the table that 

12    appears as page three, which is the first page following the 

13    text of the memorandum itself, the table that is headed, 

14    "PES state adjusted counts-unsmoothed adjustment factors." 

15               Do you see that table?

16         A.    Right.

17               (Continued on the next page) 

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2276

 1         Q.    According to Woltman, the raw PES population 

 2    figure for Maine is 1,239,554, with a standard error of 

 3    7,324?

 4         A.    Right.
                       

 5         Q.    The population estimate for Maine using the 

 6    revised post-stratification scheme is 1,256,156.  Now the 

 7    standard error has gone up to 14,327?

 8         A.    Right.

 9         Q.    That is because we now have a poststratum which 

10    is considerably smaller than the one used by the Bureau in 

11    the production PES, is that right?

12         A.    That's basically right.

13         Q.    You would agree, would you not, that differences 

14    of only one standard error are not significant?

15         A.    That's right.

16         Q.    So the original unsmoothed PES population figure 

17    for the state of Maine is within two standard errors of the 

18    Woltman estimate, is that right?  I have the calculator if 

19    you need it.

20         A.    Well, yes, and even more than you are saying is 

21    right, because there is randomness in the original adjusted 

22    count and in the revised adjusted count.  The two are also 

23    correlated, so it is not too straightforward to do the 

24    calculation of what the actual T statistic would be from the 

25    numbers we see.  But I think it is a reasonable conclusion 
                                                              2277

 1    to think that the change for Maine is not likely to be 

 2    statistically significant if Maine is taken alone.  In my 

 3    comparison I combined it with some other states and then 

 4    eventually did tests of this question over the whole of the 

 5    50 states plus the District of Columbia.

 6         Q.    I would like to look at a few more states.  I 

 7    will try not to look at another 49 of them.  How about if we 

 8    take New Hampshire, continuing with your example.  Dr. 

 9    Wachter, if you look at Mr. Woltman's chart, you see that 

10    the unsmoothed PES population figure for New Hampshire is 

11    1,115,513, with a standard error of 5,739?

12         A.    Right.

13         Q.    Then using the stratification scheme that Mr. 

14    Woltman describes in this brief memorandum, the estimate for 

15    the population becomes 1,158,185, but the standard error 

16    increases more than five times, to 32,691, isn't that 

17    correct?

18         A.    That's right.

19         Q.    Again, the original PES population figure is 

20    within two standard errors of the Woltman stratified 

21    estimate, is that right?

22         A.    Right.

23         Q.    It is the case, is it not, that the only states 

24    for which the original PES population figure is more than 

25    two standard errors from the Woltman stratified estimate are 
                                                              2278

 1    Florida, Idaho, Montana, West Virginia, and Wyoming?

 2         A.    Let's look at the table in which I give those in 

 3    my report.  Could you turn to that table?

 4         Q.    Table 5?

 5         A.    5.  The sorted T statistics which are for 

 6    Florida, Montana, Wyoming, and Idaho are T statistics 

 7    greater than 2.

 8         Q.    You did not use in your memorandum Mr. Woltman's 

 9    standard errors.  You calculated your own T statistics, is 

10    that right?

11         A.    Mr. Woltman's standard errors were inputs to my 

12    calculation.  And because he didn't give correlations, I had 

13    to do a different calculation, which I explain in the paper, 

14    to get as far as the T statistics I give here.

15         Q.    What I meant is you don't present these standard 

16    errors in the exhibit that you created for this Court, is 

17    that right?

18         A.    No, I don't re-present them.

19         Q.    So you found four states with T statistics 

20    greater than 2, is that right?

21         A.    Yes.  In the report I give the whole distribution 

22    of T statistics, and I then draw the conclusion from the 

23    distribution of the T statistics.

24         Q.    That is Table 5, is that right?

25         A.    That's right.  That is a point I tried to make in 
                                                              2279

 1    my direct testimony.

 2         Q.    Table 5 includes 52 observations -- 50 states, 

 3    the District of Columbia, and the national total -- is that 

 4    right?

 5         A.    The national total is shown here for convenience, 

 6    yes.

 7         Q.    We'll call call it 51.  A T statistic of 2 is the 

 8    rough equivalent here of significance at the .05 level, is 

 9    that fair?

10         A.    That is fair.

11         Q.    The expectation with 51 observations is that, by 

12    sheer random chance, you are going to have two or three with 

13    T statistics greater than 2, right?

14         A.    There are one-sided ways and two-sided ways of 

15    doing the test.

16         Q.    Which way did you do it?

17         A.    What I did was actually, I believe, what are 

18    called quantile plots of the distribution of the T 

19    statistics, which is a more refined way of doing the test.

20         Q.    What is the expected number of T statistics that 

21    would be greater than 2 for Table 5?

22         A.    A 5 percent level gives 1 in 20, so we have a 

23    couple of T statistics greater than 2, and there are four of 

24    them greater than 2 here.

25         Q.    The expected number would be two and a half, two 
                                                              2280

 1    or three T statistics greater than 2, is that right?

 2         A.    Yes.

 3         Q.    In fact you got four, is that right?

 4         A.    Yes.  And I also have a deficit at the other end 

 5    of the table.

 6         Q.    The result that you obtained is not statistically 

 7    significant, is that right, four T statistics greater than 2?

 8         A.    Let's see exactly what I say in my report.  No.  

 9    I found from my test that there appears to be a 

10    statistically significant excess of states with large gains 

11    from the Bureau's choice of poststrata compared to Woltman's 

12    choice.

13         Q.    I am asking what the expected number of T 

14    statistics greater than 2 would be and whether what you 

15    obtained is statistically significant.  What would the 

16    expected number be?

17         A.    What I obtained I found to be statistically 

18    significant.  You are trying to reduce what I did to a 

19    slightly more stylized version.  The version you are giving 

20    is a useful version, but the simple conclusion you are 

21    trying to draw from it can't be drawn like that.  There is 

22    some statistically significant excess, but there is not a 

23    large excess of large T statistics, and I tried to make that 

24    clear in my direct testimony.

25         Q.    Would you agree that obtaining four T statistics 
                                                              2281

 1    greater than 2 is not in itself a statistically significant 

 2    result?

 3         A.    This collection of T statistics, when compared 

 4    against the T distribution, indicates that there is a 

 5    statistically significant excess.  If the only number one is 

 6    comparing is whether there are four over 2 compared to an 

 7    expected value of two and a half over 2 out of 51, there are 

 8    certainly settings in which that wouldn't be statistically 

 9    significant.

10         Q.    Do you present evidence in this exhibit that this 

11    is a situation in which obtaining four T statistics over 2 

12    for 51 observations is statistically significant?

13         A.    I present my conclusion based on my best 

14    judgment.  I didn't make strong claims about the statistical 

15    significance.  I made weak claims about the statistical 

16    significance.  I believe that the claims that I made are 

17    justified by the numbers as they come out here.  But I 

18    didn't make strong claims because I don't think the evidence 

19    that these changes rise above the level of random noise in 

20    the PES is strong.  

21               I believe that what I say is true:  There appears 

22    to be a statistically significant excess of states.  But I 

23    point out the large sampling errors in the PES which make 

24    most of the changes in shares produced by dual-system 

25    estimation highly uncertain also make most of the changes 
                                                              2282

 1    produced by changing poststrata highly uncertain.

 2         Q.    The sampling errors that we are talking about now --

 3         A.    That's right.

 4         Q.    -- include those that, in effect, are created by 

 5    the re-post-stratification using smaller poststrata, is that 

 6    right?

 7         A.    That's right.

 8         Q.    You testified yesterday, did you not, that in 

 9    Defendants' Exhibit 44 you treat the Woltman estimates as a 

10    baseline for measuring the Bureau's adjustment?  Is that 

11    right?

12         A.    I am presenting the tables in that way.  It is 

13    confusing if you go back and forth in both directions, so I 

14    chose one direction with which to compute the tables.

15         Q.    Would it surprise you if it turned out to be the 

16    case that using whatever loss function you like, with the 

17    Woltman figures as the target, in guaging distributive 

18    accuracy the Bureau's PES-based adjusted figures have less 

19    loss than do the unadjusted census figures?  

20               MR. BARON:  I object as to form.  

21               THE COURT:  Overruled.

22         A.    One of the big issues in the loss function 

23    analysis is the heavy concentration of losses in a few 

24    states.  While the Woltman post-stratification affects those 

25    numbers, it doesn't remove that effect, and it wouldn't 
                                                              2283

 1    surprise me at all that if you put all the states together, 

 2    if you have California in there, that result comes out.  

 3    That is the result I would expect.

 4         Q.    We turn now to Defendants' Exhibit 42, another 

 5    piece that you prepared called "Evidence of Local 

 6    Heterogeneity from Further Analysis of P-12 Project Tapes."  

 7    Do you have that exhibit before you?

 8         A.    I do.

 9         Q.    The P-12 tape with which you were working and a 

10    copy of which we received yesterday afternoon focuses on 

11    proxy variables, not directly on undercount rates, is that 

12    right?

13         A.    Right.

14         Q.    The expectation underlying the Bureau's P-12 

15    project was that those variables would correlate with 

16    undercount, not be identical with it, is that right?

17         A.    That's right.

18         Q.    So the purpose of studying the proxy variables in 

19    P-12 is to determine whether post-stratification 

20    meaningfully reduces heterogeneity, not whether it achieves 

21    perfect homogeneity, is that right?

22         A.    No.  The purpose is studying it is to gauge the 

23    amount of residual heterogeneity against the other numbers 

24    in the picture, like the amount removed.  You may be able to 

25    remove lots and lots, but if there is still a lot left, you 
                                                              2284

 1    have problems.  It is not that you have gotten rid of a lot 

 2    of the trouble, it's what is the trouble left.

 3         Q.    You are looking to see whether you reduce it?

 4         A.    You are looking to see how much is left, how much 

 5    residual heterogeneity is left.

 6         Q.    That is one way of saying whether you reduced it, 

 7    to see how much is left, is that right?  If it is not right, 

 8    I will withdraw the question.

 9         A.    Yes, basically.  But what you care about is 

10    whether you have still got a problem, not whether you would 

11    have had a worse problem if you hadn't post-stratified.

12         Q.    The Bureau has been continuing to conduct 

13    research along the lines established in P-12, is that not 

14    right?  Or do you know?

15         A.    I don't have personal knowledge of that.

16         Q.    If you don't know what research they are doing, 

17    that is fine.  

18               You don't know whether any such research 

19    conducted by whomever has been presented to the Committee on 

20    Adjustment of Post-Censal Estimates?

21         A.    Dr. Thompson in a phone call once told me that 

22    they hoped to pursue studies along lines that had been 

23    pioneered by Cary Isaki and by Dr. Wolter and Dr. Causey 

24    which would use data of this kind but not use it along the 

25    lines that P-12 uses it.  He didn't present to me any 
                                                              2285

 1    details of those plans.  I really have no detailed personal 

 2    knowledge of what the CAPE committee has looked at.  

 3               MR. GOLDIN:  Your Honor, defendants have 

 4    previously moved into evidence a document identified as 

 5    Defendants' Exhibit 31, which presents the minutes of the 

 6    Committee on Adjustment of Post-Censal Estimates for April 

 7    6, 1992.  We had some very brief testimony about this 

 8    witness about it previously.  On its face that document 

 9    refers back to another set of minutes dated March 9, 1992.  

10    We have marked that set as Plaintiffs' Exhibit 742.  

11               I would simply move this document into evidence 

12    at this time.  I believe it is authenticated by virtue of 

13    defendants having produced it to us as the minutes of the 

14    CAPE, at the same time that they gave us the April 6th 

15    minutes.  I believe that by virtue of its being a product of 

16    the defendants, it is not hearsay.  

17               MR. BARON:  Your Honor, that constitutes 

18    testifying.  This document has not been authenticated.  We 

19    object to its admission.  

20               MR. GOLDIN:  Your Honor, let me, if I may, hand 

21    up a copy to the Court.  This document was provided to us by 

22    the defendants, who represented to us at the time that they 

23    provided it to us that it was, in fact, the minutes of the 

24    CAPE.  If defendants wish to withdraw that representation, 

25    your Honor, then I would ask that counsel so state at this 
                                                              2286

 1    time.  

 2               MR. BARON:  I'm sorry.  I was engaged in a 

 3    colloquy.  Could you repeat that?  

 4               MR. GOLDIN:  I was saying, Mr. Baron, that when 

 5    this document was produced to us, it was represented that 

 6    this document was, in fact, the minutes of the CAPE for 

 7    March 9, 1992.  I was asking whether you are now, on behalf 

 8    of yourself and your colleagues, withdrawing that 

 9    representation.  

10               MR. BARON:  No, I am not withdrawing that 

11    representation.  

12               MR. GOLDIN:  Then, your Honor, I believe the 

13    document is authenticated.  

14               THE COURT:  I agree.  

15               MR. BARON:  Your Honor, our objection goes more 

16    to relevance.  

17               MR. GOLDIN:  Your Honor, the defendants have put 

18    in the April 6th minutes.  The April 6th minutes 

19    cross-reference --

20               THE COURT:  The document has been authenticated, 

21    and I hold that it is relevant within the meaning of the 

22    Federal Rules of Evidence.  Plaintiffs' Exhibit 742 is 

23    therefore admitted.  

24               MR. GOLDIN:  Thank you, your Honor. 

25               (Plaintiffs' Exhibit 742 for identification was 
                                                              2287

 1    received in evidence)

 2         Q.    Yesterday, Dr. Wachter, you testified that one 

 3    bit of evidence -- 

 4               THE COURT:  How are we doing on time?  

 5               MR. GOLDIN:  I am very close to the end, your 

 6    Honor.  How am I doing in terms of what I promised earlier?  

 7               THE COURT:  Yes, that is what I am asking you.  

 8    You are over it already.  

 9               MR. GOLDIN:  A little more?  

10               THE COURT:  All right.

11         Q.    Yesterday, Dr. Wachter, you testified that one 

12    bit of evidence leading you to worry that adjustment might 

13    be the wrong thing to do was what you took to be New York 

14    City's low undercount rate as calculated by the PES.  Do you 

15    recall that testimony?

16         A.    I recall testimony along those lines.  I was 

17    discussing the effects of unreached, possible effects of 

18    unreached people.

19         Q.    New York City's undercount rate in the PES was 

20    3.0 percent, was it not?

21         A.    I don't keep the figures instantly in memory, but 

22    it was a bit below the national average -- a bit above the 

23    national average for all places, and that sounds about 

24    right.

25         Q.    The national average, in turn, was 2.1 percent, 
                                                              2288

 1    is that right?

 2         A.    That's right.

 3         Q.    Going into the census you would have expected New 

 4    York City's undercount to be higher, is that right?

 5         A.    Well, yes.

 6         Q.    Higher vis-a-vis the national average?

 7         A.    In a general way I would have expected New York 

 8    City's undercount to be well up in the range of undercounts 

 9    for big cities with centers with urban problems.

10         Q.    You suggested yesterday that perhaps what 

11    happened in New York City was that there were a lot of 

12    erroneous enumerations which canceled out a lot of gross 

13    omissions, leading to what I believe you called a 

14    self-adjusting census, is that right?

15         A.    That was one of the two possibilities that I put 

16    for barred.

17         Q.    A self-adjusting census, as you used the term, 

18    would be one in which there is a lot of census error and 

19    with some luck much of it winds up in the same places as it 

20    off sets other kinds of census error, and maybe the overall 

21    census doesn't have quite as much net undercount as you 

22    might get with the gross omissions alone, is that about 

23    right?

24         A.    That's about right.

25         Q.    It is your position, Dr. Wachter, that if that is 
                                                              2289

 1    indeed what happened in the 1990 census, a self-adjusting 

 2    census, you would prefer that to a census adjusted for 

 3    undercount with the PES?

 4         A.    The question is whether the balancing of gross 

 5    omissions by erroneous enumerations is sufficiently close in 

 6    local areas that we are really ending up with census counts 

 7    that have good numeric or distributional accuracy, that it 

 8    balances at some large aggregate level isn't enough, but --

 9         Q.    But it is your position that if we lucked out 

10    that way, that is better than adjusting with the PES?

11         A.    If we got closer to the real numbers that way, 

12    than the PES can get with synthetic adjustment, that would 

13    be better.

14         Q.    Yesterday, Dr. Wachter, you testified that you 

15    were being paid $200 per hour for your work in connection 

16    with this litigation.  How many hours have you now spent in 

17    total for which you have received or expect to receive 

18    compensation?  

19               THE COURT:  Thanks to you, they are considerably 

20    larger.  

21               MR. GOLDIN:  Doing our part, your Honor.

22         A.    I don't have a precise figure, but I think a 

23    number between 600 and 900 would be a reasonable guess at 

24    this point, when all is done.  

25               MR. GOLDIN:  I have nothing further at this time, 
                                                              2290

 1    your Honor.  

 2               THE COURT:  Thank you.  

 3               Mr. Baron, you have some brief redirect, I trust?  

 4               MR. BARON:  I do.  

 5    REDIRECT EXAMINATION 
                           

 6    BY MR. BARON:

 7         Q.    Professor Wachter, do you recall being asked 

 8    yesterday whether you signed a joint letter in 1988 top 

 9    Representative Dymalley?

10         A.    Yes

11         Q.    I have handed you a document marked Defendants' 

12    Exhibit 89.  Can you identify it?

13         A.    Yes.  This is the letter actually to the House 

14    Subcommittee on Census and Population, which I signed.

15         Q.    Can you tell the Court what the purpose of the 

16    letter was?  

17               MR. GOLDIN:  Your Honor, I will object except to 

18    the extent that the witness is being asked to testify about 

19    his own personal involvement with the letter, there being 

20    many other signatures.  

21               THE COURT:  I take that to be the question.  

22               MR. GOLDIN:  Thank you.

23         A.    The letter points out that there are awe number 

24    of problems that would be likely to be involved with census 

25    adjustment.  This is a letter from 1988, and it emphasizes 
                                                              2291

 1    -- the final conclusion is that so far the technical case 

 2    for adjustment is weak.

 3         Q.    Do you recall being asked yesterday by Mr. Goldin 

 4    whether David Freedman subsequently solicited your signature 

 5    on one or more other letters to Kirk Wolter at the Census 

 6    Bureau?

 7         A.    I remember being asked.

 8         Q.    Do you recall testifying that you did not 

 9    remember whether you had signed any one of those letters?

10         A.    That's right.  

11               MR. BARON:  May I approach?  

12               THE COURT:  Yes.

13         Q.    I have handed you, Professor Wachter, a document 

14    marked Defendants' Exhibit 90.  Can you identify this 

15    document?

16         A.    Yes.  This is one of the letters that I remember 

17    being asked to sign, a letter to Dr. Kirk Wolter.  Dr. 

18    Wolter had written to all the signatories of the first 

19    letter, including myself.  This is a reply from a number of 

20    those signatories.

21         Q.    You have seen this letter before?

22         A.    I have, yes.

23         Q.    Or a draft of this letter before?

24         A.    I think I have seen this letter before.  This is 

25    a copy of a letter I have seen before.
                                                              2292

 1         Q.    You did not sign this letter, Professor Wachter?

 2         A.    No.  I declined to sign this letter.

 3         Q.    How would you characterize Defendants' Exhibit 

 4    90, the substance of the text of this letter?

 5         A.    The letter begins, "You have written to signers 

 6    of the statement opposing adjustment of the 1990 census."  

 7    That was a characterization of this statement on census 

 8    adjustment.  I didn't feel that that was the way I regarded 

 9    the earlier letter that I had signed.  

10               I saw the earlier letter as pointing out the 

11    problems that census adjustment would face, as asserting 

12    that so far the case for technical adjustment is weak.  But 

13    I wouldn't have said that my position was wholesale 

14    opposition to 1990 census adjustment, so I declined to sign 

15    this letter.

16         Q.    Did you continue to hold these views up till the 

17    time of your nomination to the Special Advisory Panel?

18         A.    From this point on I really adopted a 

19    wait-and-see attitude.  I was skeptical that adjustment 

20    could work.  I was sensitive to the many problems and 

21    challenges it faced.  But I came to the position, let's say 

22    and see how it all goes out, what will the data show.  I 

23    think that was more my position from this period on.

24         Q.    Would it be a fair characterization to say that 

25    you were a man from Missouri on the question of adjustment, 
                                                              2293

 1    that you basically said, show me?

 2         A.    Well, not exactly.  There are two issues here.  

 3    One was a policy question.  On the policy question I believe 

 4    that it is right to have some presumption in favor of the 

 5    census as articulated in Guideline 1.  But in terms of the 

 6    assessment of the statistical evidence, I wasn't really sort 

 7    of sitting on a position and saying dislodge me.  

 8               I think I would look at all the evidence and 

 9    balance it in the light of my skeptical point of view, but I 

10    didn't feel that the burden of proof was particularly on one 

11    side or the other.  I wanted to see how the numbers came 

12    out.

13         Q.    Do you recall being asked yesterday about a 

14    sentence in your recommendation to the Secretary which is 

15    Defendants' Exhibit 39 concerning adjustment figures for 

16    Detroit and Chicago?

17         A.    I do.

18         Q.    Do you recall Mr. Goldin noting that the figures 

19    for the two cities that you gave in your recommendation were 

20    different from the figures in the June 13th press release?

21         A.    That's right, yes.

22         Q.    What explanation do you have for the differences?

23         A.    My figures are really on a better basis than the 

24    Bureau's figures, because Detroit and Chicago are lumped 

25    together into a single set of poststrata.  The homogeneity 
                                                              2294

 1    assumption here is the assumption that the people in the 

 2    same demographics in the two cities have the same undercount 

 3    rates.  

 4               My figures were computed separating Detroit from 

 5    Chicago.  While they are also subject to random variability, 

 6    such as Mr. Goldin has been pointing out, the numbers come 

 7    out for Detroit and Chicago rather strikingly different and 

 8    in the direction, in fact to the extent, that I pointed out 

 9    in my report.  

10               So this is a case, I think, where the homogeneity 

11    assumption is very likely to have introduced substantial 

12    errors into the Bureau's adjustment factors and in which my 

13    figures give a better representation of what the PES field 

14    data showed.

15         Q.    In your view, did the Secretary draw the correct 

16    inference from your recommendation on this point?

17         A.    I mention at the end of this passage that we are 

18    referring to this issue of homogeneity and heterogeneity, 

19    and the Secretary in his discussion of this passage took up 

20    that issue.  I think he did draw the correct inference from 

21    it.

22         Q.    Do you recall being asked yesterday by Mr. Goldin 

23    why you chose the sites of New York City, Detroit, Chicago, 

24    Arlington, Texas, and Berkeley for your computer 

25    simulations?
                                                              2295

 1         A.    I do.

 2         Q.    These sites were not meant to be representative, 

 3    were they?

 4         A.    No, they are not meant to be representative.  

 5    They are meant to include some cases of each of the main 

 6    settings in which the problem heterogeneity might be 

 7    important.  So I had central city areas, metropolitan areas, 

 8    and some nonmetropolitan areas that I was in the end not 

 9    able to work with, and I had some cases for each of the four 

10    census regions.  

11               So the sites were selected strategically to give 

12    me a range of important cases.  But within those decisions, 

13    whether I picked Los Angeles or New York for my central city 

14    was essentially an arbitrary decision, guided by the kinds 

15    of small factors that Mr. Goldin mentioned.

16         Q.    With respect to your computer simulations, you 

17    testified yesterday and also some today describing some of 

18    the complexities in the construction of hypothetical truth 

19    and hypothetical adjustments in these simulations, correct?

20         A.    That's right, yes.

21         Q.    How would you compare your construction of 

22    hypothetical truth in the simulations to what the Bureau 

23    did?

24         A.    The Bureau is also involved in a reasonably 

25    similar kind of construction of hypothetical truths and 
                                                              2296

 1    hypothetical adjustments in its loss function analysis.  

 2    These are not entirely the same process, but my simulations 

 3    and the Bureau's simulations are similar in many respects.

 4         Q.    With respect to your computer simulations in 

 5    Defendants' Exhibit 46, do you point out in your report that 

 6    these simulations measure relative accuracy under conditions 

 7    which assume no bias in PES estimates?

 8         A.    Yes, I believe I do point that out.

 9         Q.    Can you explain to the Court what this issue 

10    concerns?

11         A.    I tried to spell this out in my direct testimony.  

12    These simulations are really looking at the additional 

13    errors that may come in in the course of the synthetic 

14    process.  They are not measures of total error that come 

15    from all the errors in adjustment.  So I regarded the levels 

16    of disimprovement that you see in these simulations, when 

17    you think of them as extra errors rather than all the 

18    errors, as rather serious causes for concern.

19         Q.    Do you recall testifying earlier today that loss 

20    functions were personal and political?

21         A.    Yes.

22         Q.    Was your choice of loss functions political?

23         A.    The phrase "personal and political" is from my 

24    recommendation to the Secretary in which I pointed out that, 

25    in general, choices among loss functions are not scientific 
                                                              2297

 1    choices but personal and political choices.  In this case my 

 2    use of the unit counting was a personal choice, largely 

 3    guided by wanting to have a statistic that could be 

 4    understood readily for presentation.  But there were no 

 5    political considerations that went into that choice on my 

 6    part.

 7         Q.    Do you recall, Professor Wachter, discussing 

 8    presmoothing and smoothing earlier in your testimony today 

 9    and the amount of time and space that you devoted to the 

10    issue in your recommendation to the Secretary?

11         A.    I discussed it on cross-examination.

12         Q.    Prior to the conference call or call that you 

13    referred to in your testimony about the issue of 

14    presmoothing, did you foresee this to be a serious issue?

15         A.    No.  Prior to that conference call, I certainly 

16    didn't expect that the question of presmoothing or not 

17    presmoothing the variances would have any big effect on the 

18    adjusted counts.  I expected it to have small effect, make 

19    small changes in detail.  

20               I had no idea that it would turn out to be one of 

21    the major determinants of the adjusted counts.  I was quite 

22    surprised.  I shifted my attention, Dr. Hoaglin shifted his 

23    attention, all of us shifted our attention at that late 

24    moment to this new issue that had suddenly arisen.

25         Q.    Professor Wachter, would you have foreseen that 
                                                              2298

 1    changes in the presmoothing methodology affected the 

 2    apportionment of the U.S. House of Representatives?

 3         A.    No, I wouldn't have foreseen that at all.  

 4               MR. BARON:  I have no further questions.  

 5               THE COURT:  Thank you.  

 6               MR. GOLDIN:  Nothing further, your Honor.  

 7               THE COURT:  Thank you very much, Doctor.  You are 

 8    excused.  

 9               (Witness excused) 

10               MR. GOLDIN:  Your Honor, we are at the Court's 
                                                                

11    pleasure.  We can go on with our next witness.  Although I 

12    have been advised, pursuant to a side bar that we had last 

13    week with your Honor, that the plaintiffs want to move in 

14    some documents.  I think it is 31 in all.  We can either 

15    proceed with our next witness, take care of that business 

16    now, break for lunch, whatever the Court pleases.  

17               THE COURT:  We are still swimming.  Are the 

18    documents controversial?  

19               MR. MILLET:  We do have some objections.  

20               THE COURT:  Are there some of the 30 on which 

21    there are no objections?  

22               MR. MILLET:  Unfortunately, only a couple.  

23               THE COURT:  Let's just move object to the 

24    documents.  

25               MR. ZIMROTH:  Your Honor, these were the 
                                                              2299

 1    documents that we winnowed down from the hundreds that we 

 2    had originally marked.  This is what is left.  

 3               THE COURT:  All right.  

 4               MR. COHEN:  In addition to these documents, your 

 5    Honor, there are some documents marked as plaintiffs' 

 6    exhibits which will come in or which we will proffer to the 

 7    Court as part of the deposition testimony.  

 8               THE COURT:  Right.  

 9               MR. COHEN:  Right now this is just -- 

10               MR. MILLET:  We will have some in that category 

11    as well, with cross-designations.  

12               MR. COHEN:  This is just documents with no 

13    witness sponsoring them.  

14               The first one is Plaintiffs' Exhibit 45.  It is a 

15    facsimile sent from John Thompson of the statistical support 

16    division of the Bureau of the Census to Mark Plant, deputy 

17    under secretary for economic affairs on June 19, 1991.  

18    Annexed to it are two tables -- 

19               MR. MILLET:  We have no objection.  We can at 

20    least start on a good note.  

21               THE COURT:  45 is admitted. 

22               (Plaintiffs' Exhibit 45 for identification was 

23    received in evidence) 

24               MR. COHEN:  The second one, your Honor, is 

25    Plaintiffs' Exhibit 57, the report from the Panel on 
                                                              2300

 1    Decennial Census.  

 2               THE COURT:  Are you going to take these in 

 3    numerical order to make these easy?  

 4               MR. COHEN:  I am.  

 5               THE COURT:  We are at 57?  

 6               MR. COHEN:  57, Panel on Decennial Census 

 7    Methodology, Committee on National Statistics, Commission on 

 8    Behavioral and Social Sciences and Education, National 

 9    Research Council.  The name of the report is Planning the 

10    1990 Census, Priorities for Research and Testing, an Interim 

11    Report."  

12               MR. MILLET:  Your Honor, we do object to this 

13    one.  There is no foundation for it.  On its face it is 

14    labeled an interim report.  There were witnesses present in 

15    this courtroom who are listed as members of the group who 

16    could have laid a foundation presumably if one was able to 

17    be laid.  So we object to lack of foundation.  

18               MR. COHEN:  Your Honor, let me mention that 

19    serving on this panel were, among others, Mr. Rolph.  You 

20    have heard testimony during the course of the trial about 

21    the activity of the panel from Mr. Rolph, from Professor 

22    Fienberg, from Dr. Bailar and Dr. Wolter, the cooperation 

23    between the Bureau and the panel in the production of 

24    several reports.  

25               This is one of the reports on the way to the 
                                                              2301

 1    report of 1985 on the decennial census.  I think it would be 

 2    useful to the Court to see the planning and preparation that 

 3    went into the production of the final report that has 

 4    already been admitted into evidence.  

 5               THE COURT:  It is relevant.  It is authentic.  

 6    The only argument that it is interim.  It is admitted, 

 7    Plaintiffs' Exhibit 57.  

 8               (Plaintiffs' Exhibit 57 for identification was 

 9    received in evidence) 

10               MR. COHEN:  79 and 80, your Honor, are both 

11    memoranda from Professor Tukey to staff of the Census 

12    Bureau, involving some of his thinking on smoothing.  They 

13    came up, I'm sorry I don't have a date ready at hand.  One 

14    was in August and the other with you was in September of 

15    1990.  They were advice from Professor Tukey on some of the 

16    thinking about smoothing that went into the process of 

17    producing the production.  

18               MR. MILLET:  Where is that date?  

19               MR. COHEN:  You can see it at the top.  

20               MR. MILLET:  You are referring to the fax date?  

21               MR. COHEN:  The fax date, yes.  They were 

22    preliminary to the ultimate determination made by Census 

23    Bureau.  I think that they are quite relevant to the issues 

24    that are involved in this case.  

25               MR. MILLET:  Your Honor, we do object to both of 
                                                              2302

 1    these on similar grounds.  There is no foundation for their 

 2    admission.  Your Honor, Dr. Tukey was here present in the 

 3    courtroom.  If they wanted to have a discussion about these, 

 4    they should have done it while he was here on direct.  

 5               THE COURT:  Why didn't you raise this with Dr. 

 6    Tukey?  

 7               MR. COHEN:  There has been a good deal of 

 8    testimony about Dr. Tukey's conversations, including 

 9    testimony this morning from Professor Wachter about a 

10    conversation on May 16, 1991, involving some of the issues 

11    concerning smoothing.  We did not think this would be 

12    controversial.  They are memoranda from Dr. Tukey to Cary 

13    Isaki, Paula Schneider, and John Thompson.  Mr. Ttompson and 

14    Ms. Schneider were members of the Bureau's Undercount 

15    Steering Committee, and Cary Isaki was the person at the 

16    Bureau chiefly responsible some of the smoothing work.  

17               MR. MILLET:  Your Honor, we are being deprived of 

18    an opportunity to cross-examine Dr. Tukey on these 

19    documents.  

20               THE COURT:  I am going to sustain the objection 

21    to both of them.  

22               MR. MILLET:  We have no objection to 96.  

23               THE COURT:  96 is admitted without objection.  

24               (Plaintiffs' Exhibit 96 for identification was 

25    received in evidence) 
                                                              2303

 1               MR. COHEN:  PX-149, your Honor, is a memorandum, 

 2    a report, produced by employees of the Census Bureau, David 

 3    Bateman, John Clark, Mary Mulry, and John Thompson, 

 4    discussing the 1990 post-enumeration survey evaluation 

 5    results.  We have heard a good deal of testimony these last 

 6    two weeks about the P studies.  This is highly relevant to 

 7    the testimony that has come before your Honor.  

 8               MR. MILLET:  The authentication is being provided 

 9    by Mr. Cohen.  Your Honor, this document is not dated.  I 

10    believe there is also some handwriting on this document 

11    which has not been authenticated.  I would note that the 

12    footnote of acknowledgement on the last page indicates that 

13    this is not to be taken as the views of the Census Bureau.  

14               MR. COHEN:  We will offer it without the 

15    handwriting.  

16               MR. MILLET:  If it is being offered as some of 

17    the of the P studies, I would remind the court that we did 

18    have a witness prepared to discuss the P studies, and Dr. 

19    Breiman was not allowed to testify.  

20               MR. COHEN:  I would think that counsel would want 

21    to hear the views of the Census Bureau on the P studies.  

22               MR. MILLET:  The views of the Census Bureau on 

23    the P studies are in the record.  

24               MR. COHEN:  This is a matter highly relevant to 

25    the determinations that the Court is going to be making in 
                                                              2304

 1    the coming days.  

 2               THE COURT:  These are the thoughts of people who 

 3    worked for the Census Bureau who were not called as 

 4    witnesses and whom the Census Bureau has expressly put in 

 5    the report that they do not take a position on the views of 

 6    these people.  I am going to sustain the objection to 149.  

 7               MR. COHEN:  May I make just one comment, your 

 8    Honor?  

 9               THE COURT:  Yes. 

10               MR. COHEN:  There is a standard caveat that the 

11    Census Bureau places on all its documents.  Indeed it is, 

12    however, our understanding that notwithstanding that 

13    disclaimer, these are documents produced under the auspices 

14    of the Census Bureau, put out by the Census Bureau.  You 

15    will find it in -- 

16               THE COURT:  When you say "put out," what does 

17    that mean?  

18               MR. COHEN:  That means these are the reports that 

19    the Census Bureau issues following studies conducted by 

20    Census Bureau employees.  They always put a caveat at the 

21    end, presumably for purposes just like this, so they can be 

22    disowned at the time of litigation.  

23               MR. MILLET:  Your Honor, that is a highly 

24    objectionable statement.  Mr. Cohen is the one providing the 

25    authentication of this.  That is improper.  The document on 
                                                              2305

 1    its face is as your Honor characterized it.  We object to 

 2    it.  

 3               THE COURT:  I am going to stick with my ruling on 

 4    149.  

 5               MR. MILLET:  Your Honor, we don't object to the 

 6    next one, which I believe is 163, although I believe it is 

 7    duplicative of one of the earlier exhibits.  It is the same 

 8    text as -- 

 9               THE COURT:  In case it isn't, it is admitted as 

10    Plaintiffs' Exhibit 163.  

11               Plaintiffs' Exhibit 163 for identification was 

12    received in evidence) 

13               MR. COHEN:  Your Honor, 168 are several pages of 

14    transcripts from the trial in Cuomo v. Baldridge.  I believe 

15    it was the 1984 trial.  It is the testimony of Dr. Passell, 

16    whose then a Census Bureau employee.  They are admissions of 

17    the Census Bureau.  I don't believe that any further 

18    authentication is needed.  

19               MR. MILLET:  We do object on hearsay grounds.  

20    Certainly, Dr. Passell was an employee of the Census Bureau, 

21    as are thousands of others.  But not every statement by an 

22    employee, particularly statements made on cross-examination 

23    in the middle of trial, can be said to have been authorized 

24    previously by the Census Bureau.  

25               It is hearsay.  It is not within Rule 803 because 
                                                              2306

 1    there has been no showing that as to Dr. Passell's 

 2    availability or unavailability.  

 3               THE COURT:  Do we know whether he is available or 

 4    not?  

 5               MR. MILLET:  He is not employed by us any longer.  

 6               MR. COHEN:  Dr. Passell was called by the Census 

 7    Bureau in 1984.  

 8               THE COURT:  Where is he today?  

 9               MR. MILLET:  I believe he is at the Urban 

10    Institute.  That is a group --

11               MR. COHEN:  In Washington.  

12               MR. MILLET:  He has worked with the plaintiffs' 

13    panel members and provided a report to plaintiffs' panel 

14    members during the course of this.  

15               I would also note, as many of plaintiffs' own 

16    witnesses have said, the issues in the 1984 trial were a 

17    great deal different than they are here.  

18               THE COURT:  What are you putting it in for?  Is 

19    there some particular sentence in here you want me to look 

20    at?  

21               MR. COHEN:  Yes.  It is relevant, your Honor, to 

22    the issue of correlation bias and where those who are missed 

23    in both the census and the PES are likely to be found.  In 

24    fact, it talks about evidence pertaining to illegal aliens, 

25    the evidence adduced at trial in 1984.  
                                                              2307

 1               THE COURT:  You are talking about 1037 at lines 

 2    8, etc.?  

 3               MR. COHEN:  Yes.  

 4               THE COURT:  Haven't you got enough in evidence 

 5    without this?  

 6               MR. COHEN:  If your Honor is prepared to rule in 

 7    favor on that, we will leave this out.  

 8               THE COURT:  I don't think I will do that.  

 9               MR. COHEN:  Exhibit 169, your Honor, is a 

10    memorandum from Mr. Passell while an employee of the Census 

11    Bureau in 1985.  Again it is bearing on the location and the 

12    extent of undocumented aliens and the effect that that has 

13    on the issue of correlation bias.  

14               MR. MILLET:  We object to the lack of foundation, 

15    your Honor.  This is on its face a 1985 document.  I have no 

16    way of knowing from the face of this whether this is a final 

17    document, whether it is a draft document, whether Dr. 

18    Passells' view has changed at any point.  There is simply no 

19    foundation for this.  

20               THE COURT:  Was he working for the Census Bureau 

21    when he wrote this?  

22               MR. COHEN:  He was, your Honor.  

23               THE COURT:  I will admit Plaintiffs' Exhibit 169.  

24               (Plaintiffs' Exhibit 169 for identification was 

25    received in evidence) 
                                                              2308

 1               MR. COHEN:  You will be glad to know, your Honor, 

 2    that Plaintiffs' Exhibit 170 is a paper by Mr. Passell, 

 3    published in a journal of learned papers, Demography, volume 

 4    24, number 3.  It is also on the issue of the extent and 

 5    location of the undocumented in the 1980 census.  

 6               MR. MILLET:  We object to the lack of foundation.  

 7    I note on the face of this document that it appears 

 8    irregular.  It appears to be two documents put together.  It 

 9    certainly doesn't all appear to be taken from a learned 

10    journal.  There are two totally different typefaces.  I 

11    don't know what this document is.  

12               MR. COHEN:  The second is, I believe, a draft.  

13    I'm sorry.  It is the appendix to the article itself.  

14               THE COURT:  The appendix?  

15               MR. MILLET:  It certainly appears different, 

16    Judge.  

17               THE COURT:  It does look as though two separate 

18    documents are here. 

19               MR. MILLET:  I also note, your Honor, there are 

20    learned treatises under the hearsay rule certainly can be 

21    reviewed, but they are generally not admitted as evidence.  

22               THE COURT:  You don't care if I just take it to 

23    review it?  

24               MR. MILLET:  It is not competent as evidence, 

25    your Honor, at least without a foundation.  I think on the 
                                                              2309

 1    face of the document, it appears irregular.  

 2               THE COURT:  But not sufficiently irregular to 

 3    bother me.  Plaintiffs' Exhibit 170 is admitted.  

 4               (Plaintiffs' Exhibit 170 for identification was 

 5    received in evidence) 

 6               MR. COHEN:  Finally, your Honor, we have a paper 

 7    by a Mr. Passell, "The Geographic Distribution -- 

 8               THE COURT:  You have a passel of documents here. 

 9               MR. COHEN:  I object wish I had thought of that.  

10               THE COURT:  You're not Irish.  

11               MR. COHEN:  I'm told my name is actually Qqinn 

12    and I'm from Coney Island.  

13               THE COURT:  Neighbors.  

14               MR. COHEN:  "Geographic Distribution of 

15    Undocumented Immigrants" on the same subject matter, again, 

16    in a journal.  I don't think I have to belabor the point.  

17               MR. MILLET:  I won't either.  Let the record note 

18    my objection on the same grounds.  

19               THE COURT:  171 is admitted.  

20               (Plaintiffs' Exhibit 171 for identification was 

21    received in evidence) 

22               MR. COHEN:  Plaintiffs' Exhibit 205, your Honor, 

23    is an information memorandum for the Secretary, Secretary 

24    Klutznick, in 1980, produced by Courtenay Slater, who was 

25    then the chief economist for the Department of Commerce, and 
                                                              2310

 1    Vincent Barabba, who was then the Director of the Census 

 2    Bureau.  

 3               It is information to the Secretary explaining 

 4    their views on who ought to make the decision on adjusting 

 5    the census in 1908 and the reasons for that recommendation.  

 6    It was identified as a deposition document at the deposition 

 7    of Secretary Klutznick in November of 1980.  I don't believe 

 8    there is any dispute to the authenticity of this document.  

 9               MR. MILLET:  I haven't seen any evidence of its 

10    authenticity, and certainly in this case.  I would note, 

11    your Honor, that this particular document is not dated, it 

12    is not signed.  I don't believe there has been a sufficient 

13    foundation for its admissibility.  I would also note our 

14    relevance objections to documents going to prior adjustment 

15    decisions.  

16               THE COURT:  Prior to what?  

17               MR. MILLET:  Prior to the adjustment of the 1990 

18    decennial census.  

19               THE COURT:  This is the 1980?  I had missed that 

20    point.  

21               MR. COHEN:  This is relevant to the issue of 

22    delegation, among others, your Honor.  The past practice of 

23    the commerce department has been to delegate the decision of 

24    adjustment to the Director of the Census Bureau.  In 1987 

25    and again in 1990 the commerce department wrenched that 
                                                              2311

 1    decision from the Census Bureau and made it itself.  We 

 2    think it is useful to the Court to see how this issue has 

 3    been treated in the past.  

 4               In addition, I have the deposition that we took 

 5    of Secretary Klutznick on November 11, 19830, in which the 

 6    Secretary identifies this document.  I hand it over to Mr. 

 7    Millet if he would like to see it.  It does come out of a 

 8    prior litigation on the issue of adjustment.  

 9               MR. MILLET:  I didn't realize that one decision 

10    amounted to a practice.  I maintain my objection.  

11               THE COURT:  Sustained.  

12               MR. COHEN:  Plaintiffs' Exhibit 217, your Honor, 

13    is a memorandum from a staff assistant Barnett Denton, the 

14    statistical methods division of the Census Bureau, to Mr. 

15    Morton Boyce and explaining the national vacancy check for 

16    1970.  We have heard testimony about the kind of statistical 

17    adjustment that was conducted -- 

18               THE COURT:  May I interrupt you?  Mr. Bowker, we 

19    seem to be missing 217.  

20               You can continue talking, Mr. Cohen.  

21               MR. COHEN:  We heard testimony from, among 

22    others, Dr. Bailar and Dr. Wolter, and I believe Dr. 

23    Ericksen, about the statistical processes that were 

24    undertaken in 1970 to add a number of persons to the 1970 

25    count.  It bears on the issue of whether, among other 
                                                              2312

 1    issues, whether an adjustment in 1990 in fact would be an 

 2    aban conment of a 200-year tradition as is asserted in the 

 3    Secretary's decision.  

 4               MR. MILLET:  Your Honor, we object to the lack of 

 5    foundation for authentication and hearsay.  On its face I 

 6    point out the document has handwriting.  To me the document 

 7    certainly appears irregular on its face.  It appears to be 

 8    two documents put together.  The xeroxing looks a bit odd, 

 9    particularly the second document I appears as though there 

10    is something cut off or something strange in the xeroxing.  

11    We would maintain our objection for lack of foundation.  

12               THE COURT:  217 is admitted.  

13               (Plaintiffs' Exhibit 217 for identification was 

14    received in evidence) 

15               MR. MILLET:  We have no objection to 228.  

16               THE COURT:  It is admitted.  

17               (Plaintiffs' Exhibit 228 for identification was 

18    received in evidence) 

19               MR. MILLET:  Your Honor, I have no objection to 

20    240 if Mr. Cohen will, and I don't have Mr. McGehee's 

21    deposition memorized, if he will represent that this was 

22    identified by Mr. McGehee in his deposition as his, then I 

23    have no objection.  

24               MR. COHEN:  It was indeed, your Honor.  

25               THE COURT:  240 is admitted.  
                                                              2313

 1               (Plaintiffs' Exhibit 240 for identification was 

 2    received in evidence) 

 3               MR. COHEN:  Your Honor, Plaintiffs' Exhibit 243 

 4    is a memorandum from Mr. Monsour of the Census Bureau to 

 5    John Thompson, commenting on some of the preliminary 

 6    guidelines issued by the commerce secretary.  It is relevant 

 7    to, in particular, the issue of whether block-level accuracy 

 8    can, in fact, be assessed by the Census Bureau.  

 9               You will note in paragraph 1 Mr. Monsour 

10    commented that there is no way that the adjusted counts can 

11    be shown to be more accurate than the census at census block 

12    levels and probably not at local levels.  It is relevant to 

13    the test that the Secretary in fact applied in determining 

14    whether the adjusted counts were more accurate than the 

15    unadjusted.  

16               MR. MILLET:  We object to lack of foundation.  I 

17    particularly point out there is some handwriting on there 

18    that hasn't been identified.  

19               MR. COHEN:  This was produced to us in the course 

20    of discovery.  It says from Tortora, box 3.  Mr. Tortora, 

21    Dr. Tortora, is an employee of the Census Bureau.  In fact, 

22    he is a member of the Undercount Steering Committee.  

23               THE COURT:  243 is admitted. 

24               (Plaintiffs' Exhibit 243 for identification was 

25    received in evidence)
                                                              2314

 1               MR. COHEN:  Plaintiffs' Exhibit 276, your Honor, 

 2    is a report published by the Department of Commerce 

 3    concerning the coverage of the population in the 1970 

 4    census.  It is relevant to the issue of a longstanding 

 5    problem of differential undercount and undercount in 

 6    censuses in this country.  It is an official report.  

 7               MR. MILLET:  If the Court would just note our 

 8    prior objections to the relevant.  Other than that, we have 

 9    no objection.  

10               THE COURT:  I will note it.  It is admitted as 

11    276.  

12               (Plaintiffs' Exhibit 276 for identification was 

13    received in evidence) 

14               MR. MILLET:  On 355 I would again note a 

15    relevance objection, and that is all.  

16               THE COURT:  Why is it relevant, Mr. Cohen?  

17               MR. COHEN:  Your Honor, this is a communication 

18    from Professor Ericksen to the liaison at Census Bureau with 

19    the Special Advisory Panel.  Professor Ericksen is asking 

20    for particular information relevant to the studies of the 

21    Special Advisory Panel.  It is a letter that seeks 

22    particular information at a point before the Secretary's 

23    decision was made.  

24               I think it is relevant to the issue of whether 

25    full cooperation was provided to all the members of the 
                                                              2315

 1    Special Advisory Panel.  We heard testimony from Dr. 

 2    Ericksen that, in fact, full cooperation wasn't always 

 3    provided.  

 4               MR. MILLET:  Dr. Ericksen's report did indicate 

 5    to the Secretary that he had all the information he needed.  

 6    This document also gives Dr. Ericksen's opinion as to when 

 7    and how the Special Advisory Panel can be dissolved.  I 

 8    don't think Dr. Ericksen is qualified to give a legal 

 9    opinion.  

10               THE COURT:  355 is admitted.  

11               (Plaintiffs' Exhibit 355 for identification was 

12    received in evidence) 

13               (Continued on next page) 

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2316

 1               MR. COHEN:  383, your Honor, is a report, a paper 

 2    presented by Charles Jones, who is the associate director of 

 3    the Census Bureau for decennial censuses, in fact, the 

 4    person at the Bureau in charge of decennial censuses, on 

 5    what was learned, the lessons learned in taking the 1990 

 6    census.  

 7               We think it certainly is relevant to the issues 

 8    in this case about the success of certain automation 

 9    changes, management controls and other operations that were 

10    instituted during the census and that the court will be 

11    assisted in learning from the head of the census what, in 

12    fact, lessons they drew from their experience.  

13         MR. MILLET:  We object to the lack of foundation.

14               I don't know whether this is the final document 
                                                                 

15    or not.  Mr. Jones was deposed over a period of several 

16    days.  They could have shown this to him, they could have 

17    asked him, they could have authenticated it.

18               That is the basis of my objection.  

19               THE COURT:  I will take it.  383 is admitted.

20               (Plaintiff's Exhibit 383 marked for 

21    identification was received in evidence.) 

22               MR. COHEN:  384, your Honor, is a stipulation of 

23    facts entered into between the defendants, including 

24    Secretary Mosbacher, and the Commonwealth of Massachusetts 

25    in litigation involving the conduct of the census in 1990, 
                                                              2317

 1    specifically, litigation involving the method for counting 

 2    overseas residents as part of the population of the United 

 3    States.

 4               It is particularly pertinent, because the 

 5    stipulation recites that those counted overseas, though 

 6    inserted into individual state counts, are not, in fact, 

 7    placed in particular blocks or locations within states.

 8               From our perspective, this is an inconsistency in 

 9    the way that the Commerce Department has treated the 

10    adjustment counts under the PES adjustment and we think it 

11    is pertinent to your Honor's consideration whether, in fact, 

12    the Commerce Department is treating all census operations 

13    evenhandedly.  

14               MR. MILLET:  Your Honor, we do object to this.

15               If you can look at the first line of it, it says 

16    that this was a stipulation for purposes of that litigation 

17    only, not this case.

18               We certainly are not collaterally estopped from 

19    anything this this case.  The issues in the Massachusetts 

20    case, at least as far as I understand them, I have not been 

21    involved in that case, are quite a bit different from this 

22    one.  

23               If they want to present testimony about this 

24    issue, fine, but this is a stipulation in another case which 

25    was reached with other considerations in mind.  It applies 
                                                              2318

 1    only to that case and we object to its use here.  

 2               THE COURT:  The objection is overruled.  384 is 

 3    admitted.

 4               (Plaintiff's Exhibit 384 marked for 

 5    identification was received in evidence.) 

 6               MR. COHEN:  Your Honor, Plaintiff's Exhibit 385 

 7    is a hearing before the House Subcommittee on Census and 

 8    Population.  That is, in effect, the oversight committee.  

 9               THE COURT:  You object to this, Mr. Millet?  

10               MR. MILLET:  Yes, I do, your Honor.  

11               THE COURT:  On what grounds?  

12               MR. MILLET:  On the grounds of hearsay and also 

13    on the grounds that it includes testimony by plaintiffs' 

14    counsel and if plaintiffs' counsel wish to be witnesses in 

15    the case then they should be sworn and take the stand.  

16               THE COURT:  Who are we talking about? 

17               Oh, I see, Mr. Rifkind.  

18               MR. MILLET:  Mr. Rifkind.  

19               If this is introduced, then we would have to 

20    consider whether they can continue in this litigation or 

21    whether they would be disqualified.  

22               MR. COHEN:  Disqualified for testifying in this 

23    case?  

24               MR. RIFKIND:  Your Honor, would you like to 

25    testify before Congress?  
                                                              2319

 1               THE COURT:  Boy, if I had known.  

 2               (Laughter)

 3               MR. COHEN:  Your Honor, we will gladly delete the 

 4    testimony of Mr. Rifkind from this document.  

 5               MR. MILLET:  Then what testimony in this document 

 6    is being offered?  

 7               MR. COHEN:  There are comments by a number of 

 8    representatives, as well as by Mr. McGehee, who is a member 

 9    of the special advisory panel; Professor Tukey, Dr. Wachter.  

10               MR. MILLET:  Professors Tukey or Ericksen, who I 

11    see listed here, they had an opportunity to say it here, 

12    they had an opportunity to cross-examine Professor Wachter, 

13    and they deposed Mr. McGehee and Mr. Plant.  

14               MR. COHEN:  You will find Deputy Under Secretary, 

15    Mark Plant and the Director of Census Bureau appeared at 

16    this hearing, your Honor, and I think those witnesses who 

17    are, in fact, not witnesses in these proceedings, might have 

18    something that they said bears on the determinations that 

19    are to be made here.  

20               THE COURT:  I am going to exclude 385.  

21               MR. COHEN:  Your Honor, Plaintiff's Exhibit 499 

22    is a memorandum prepared by Susan Beverage of the 

23    statistical support division reporting in part of the 

24    Bureau's prepare series memorandum, preliminary research and 

25    evaluation memoranda, on the results of the 
                                                              2320

 1    parolee-probationer coverage program.

 2               We heard testimony from Dr. Ericksen, Mr. 

 3    Bounpane and others about the institution of this program as 

 4    a new device for 1990.

 5               This shows some of the results, the 

 6    effectiveness, or lack thereof, of that program.  

 7               MR. MILLET:  Your Honor, my objection is well 

 8    stated by the note at the bottom of the first page.  It says 

 9    the report is preliminary, tentative, cautions researchers 

10    not to rely on it, it cautions that it is not reviewed by 

11    the Bureau and should not be treated in the same way as 

12    other Bureau documents.  

13               THE COURT:  Objection sustained.  Exhibit 499 is 

14    excluded.  

15               MR. COHEN:  Plaintiff's Exhibit 556, your Honor, 

16    is a memorandum prepared by Barbara Bailar at the time that 

17    she was employed by the Census Bureau, I believe was the 

18    associate director of statistical methodology, reporting on 

19    recommendations made by the census advisory committee of the 

20    American Statistical Association, specifically with respect 

21    to plans for 1990 census adjustment research.  

22               THE COURT:  What is it relevant to?  

23               MR. COHEN:  Her testimony from Dr. Bailar, Dr. 

24    Wolter, among others, about the long years of research and 

25    cooperations between the Bureau and several of the advisory 
                                                              2321

 1    committees in planning for 1990 research programs.  

 2               THE COURT:  This does what, corroborates that?  

 3               MR. COHEN:  It, in effect, corroborates it.  

 4               THE COURT:  I am going to exclude it on the 

 5    grounds it is cumulative.  

 6               MR. COHEN:  564, your Honor, are minutes of the 

 7    census advisory committee meeting in April 1987 at a point 

 8    when much of the research into adjustment methodology was 

 9    being summed up at the Census Bureau and being presented to 

10    these advisory committees.  

11               I think it is relevant to show the reaction that 

12    those committees had to the work that the Bureau had engaged 

13    in for the better part of the decade leading up to 1987.  It 

14    is relevant to the decisions made by the Bureau in the 

15    spring of 1987 to proceed with plans for adjustment of the 

16    1990 census.           

17               MR. MILLET:  Your Honor, we object to lack of 

18    foundation, particularly this is hearsay within hearsay 

19    within hearsay, and there is also something attached to the 

20    back of it that does not appear to be complete.  

21               MR. COHEN:  Of course, these are advisory 

22    committees established by the Census Bureau, your Honor.  

23               THE COURT:  Established --  

24               MR. COHEN:  By the Census Bureau.  

25               MR. MILLET:  It doesn't make it nonhearsay.  It 
                                                              2322

 1    is the minutes of what someone else said in a meeting.  

 2               THE COURT:  I will admit it.  564 is admitted. 

 3               (Plaintiff's Exhibit 564 marked for 

 4    identification was received in evidence.) 

 5               MR. COHEN:  I don't think there should be any 

 6    problem with this one, your Honor.  

 7               THE COURT:  569 is admitted without objection.  

 8               MR. MILLET:  No, there is really a very strong 

 9    objection.  

10               MR. COHEN:  I'm sorry, I didn't mean to mislead 

11    you.

12               569 is a newspaper article or, in fact, a 

13    reproduction of a newspaper article published in the 

14    Washington Post reporting on the fact that Wendell Willkie, 

15    the second general counsel of the Commerce Department, was 

16    about to be named as the deputy chairman of the Republican 

17    National Committee.

18               There is testimony which we sought to elicit in 

19    Mr. Willkie's deposition to the effect that he had, in fact, 

20    made an announcement in 1989 at the Commerce Department that 

21    he was about to leave to become a high level official of the 

22    RNC.  That testimony was blocked by defendants' counsel at 

23    that point.

24               We are offering this as part of our case showing 

25    the political interest and intersections of some of the 
                                                              2323

 1    people who are involved in making the decision.  

 2               THE COURT:  How do you get over the hearsay 

 3    objection?  

 4               MR. COHEN:  I think Mr. Rifkind has some advice 

 5    to give me.  

 6               MR. ZIMROTH:  I think we said it's just like 

 7    Martindale Hubbell, your Honor.  

 8               THE COURT:  The Washington Post?  

 9               MR. ZIMROTH:  Probably more reliable than 

10    Martindale Hubbell.  

11               MR. MILLET:  Particularly the now famous unnamed 

12    sources which I point out.  

13               THE COURT:  The objection to 569 is sustained.  

14               MR. COHEN:  573, your Honor --

15               THE COURT:  "Dear Republican Colleague." 

16               Go on.  

17               MR. COHEN:  Is a letter from the Republican task 

18    force on reapportionment, Representative Wynn Webber 

19    commenting on that conference's position with respect to 

20    adjustment or potential for adjustment of the 1990 census.

21               I think it is a fact relevant to the testimony 

22    that Professor Cain offered earlier in these proceedings on 

23    whether, in fact, those in the Republican Party knew or 

24    ought to have known or could easily have known what effect 

25    the adjustment would have had on the prospects of Republican 
                                                              2324

 1    success following reapportionment and redistricting in 1990.  

 2               MR. MILLET:  Your Honor, my copy of this has 

 3    three different documents, none of which have been 

 4    authenticated and all of which have hearsay, including some 

 5    apparent newspaper article at the end.  The question of what 

 6    congressmen may have known or discussed is not terribly 

 7    relevant, unless it can be tied into the Secretary.  

 8               THE COURT:  I agree.  I am going to keep 573 out.  

 9               MR. COHEN:  574, your Honor, is a document better 

10    known in this litigation than I think in any other.  It is, 

11    in fact, recited in your first decision in April of 1990 on 

12    defendants' summary judgment motion or motion to dismiss.  

13    It is a memorandum from Harry Scarr, executive assistant for 

14    statistical affairs, to the then Under Secretary Ortner.

15               Mr. Scarr is still the executive assistant for 

16    statistical affairs to now acting Under Secretary Mark Plant 

17    and was one of the --  

18               MR. MILLET:  He is still there.  

19               MR. COHEN:  He is still in their office.

20               One of those who was a drafter of the Secretary's 

21    decision.

22               You may recall, your Honor, that Mr. Scarr in 

23    this memorandum opines on some of the problems with 

24    adjustment that might be faced and does so to, ostensibly to 

25    advise then Under Secretary Ortner on how Under Secretary 
                                                              2325

 1    Ortner might go about determining whether to support or 

 2    oppose adjustments.  

 3               THE COURT:  Mr. Millet.  

 4               MR. MILLET:  Your Honor, subject to our overall 

 5    objection on relevance, I would just like the record to 

 6    reflect that and also to note that we are proving deposition 

 7    pages from Dr. Scarr's deposition to put this in the 

 8    appropriate context.  

 9               THE COURT:  I will take it.  574 is admitted.

10               (Plaintiff's Exhibit 574 marked for 

11    identification was received in evidence.) 

12               THE COURT:  578.  

13               MR. COHEN:  578, your Honor, is a report of the 

14    Gallup poll on American attitudes towards adjusting the 

15    census.

16               There is a section of the Secretary's decision, 

17    as I'm sure you are aware, that concerns whether or not 

18    adjustment in 1990 will affect participation in future 

19    censuses.  We think this is relevant to that issue.  

20               MR. MILLET:  We would object on authentication 

21    and hearsay, and unless this can be tied up to having been 

22    presented to the Secretary, then I don't think it is 

23    relevant, either.  

24               MR. COHEN:  In fact --  

25               THE COURT:  I haven't read this, but does it 
                                                              2326

 1    purport to be a summary of a poll conducted by the Gallup 

 2    organization?  

 3               MR. COHEN:  Yes, it does.

 4               In fact, it is recited in the recommendations of 

 5    the special advisory panel member Tarrance, your Honor.  

 6    This is the --  he makes mention of this in his 

 7    recommendation.  This is just the underlying summary of 

 8    that.  

 9               THE COURT:  I will admit it, 578.

10               (Plaintiff's Exhibit 578 marked for 

11    identification was received in evidence.) 

12               MR. COHEN:  582, your Honor, is a reproduction of 

13    a newspaper article from the Washington Times reporting on a 

14    speech made by President Bush to governors in which the 

15    president stressed the importance of the Republican Party 

16    taking control of state houses in 1990.

17               I am advised we will withdraw it.  

18               THE COURT:  Good.  It wouldn't have gone in any 

19    way.  

20               MR. COHEN:  I was kind of close, wasn't I?  

21               THE COURT:  No, no.

22               There is another hearing, 6/10.  Let's see who is 

23    testifying here; Mr. Zimroth?  

24               MR. COHEN:  I think Mr. Zimroth was a babe in the 

25    woods in those days on the census issue.  
                                                              2327

 1               MR. ZIMROTH:  As many others.  

 2               MR. COHEN:  This is testimony of Dr. Bailar, 

 3    which Dr. Bailar discusses, I believe, the strategy that 

 4    your Honor has heard, some testimony about this thus far.  

 5               MR. MILLET:  We do object on hearsay grounds.  

 6    Dr. Bailar was here and was able to testify.  If they wanted 

 7    to put this in they should have put it in while she was here 

 8    and given us an opportunity to cross-examine her on it.  

 9               THE COURT:  610 I will take.

10               (Plaintiff's Exhibit 610 marked for 

11    identification was received in evidence.) 

12               THE COURT:  This, I believe, is the last 

13    document?  

14               MR. MILLET:  I hope it is.  

15               MR. COHEN:  Until we think of some others.  

16               623 is a memorandum from Marshall Turner of the 

17    Census Bureau to Kirk Wolter and Charles Jones, Barbara 

18    Bailar, all then of the Census Bureau, reporting on a 

19    meeting Mr. Turner had with Tom Hoffler, who is of the 

20    Republican National Committee, concerning technical issues 

21    involving adjustment of the 1990 census counts.

22               It is a memorandum of January 1987, in the period 

23    before the first dec ision on whether or not to adjust the 

24    1990 census was made, and I think it is relevant to the 

25    issues of whether extraneous influences were brought to bear 
                                                              2328

 1    either on that decision or the decision of 1991.  

 2               MR. MILLET:  Certainly, your Honor, I think both 

 3    sides are designating deposition transcripts that indicate 

 4    that the Census Bureau provided briefings to members of both 

 5    political parties when they wanted.

 6               My objection to this particular document are 

 7    evidentiary ones relating to hearsay and lack of 

 8    authentication, and, frankly, looking at the face of it, I'm 

 9    not sure this is complete.  I don't know whether there were 

10    more pages to this or not.  

11               MR. COHEN:  I don't know of any other pages, your 

12    Honor.  

13               THE COURT:  Who is Turner?  

14               MR. COHEN:  Marshall Turner is in the part of the 

15    Census Bureau that acts as liaison to state data centers.  

16    In effect --  

17               THE COURT:  So this is a memo from Mr. Turner, 

18    who works in the Census Bureau, to several other people who 

19    are within the Census Bureau?  

20               MR. COHEN:  That's correct, all are Census Bureau 

21    employees.  

22               THE COURT:  I will admitt it.  610 is admitted.

23               (Plaintiff's Exhibit 610 marked for 

24    identification weighs received in evidence.) 

25               MR. MILLET:  We have one.  
                                                              2329

 1               MR. BARON:  One additional document, your Honor.

 2               In my cross-examination of Dr. Rolph, I 

 3    identified an exhibit marked 516.  The identification was at 

 4    page 858 of the transcript of these proceedings.  It was a 

 5    document that the witness, Dr. Rolph, identified himself, he 

 6    recognized as a Rand note authored by him entitled, "Using 

 7    statistical tools," dated April 1984.

 8               I did not move it in evidence at that time.  I 

 9    ask that it be moved in now.  

10               MR. RIFKIND:  Do you have another copy?  

11               MR. BARON:  I do not have another copy.  It is 

12    516.

13               (Handing to Mr. Rifkind.) 

14               (Pause) 

15               MR. BARON:  You recall I asked Dr. Rolph a 

16    hypothetical question, it was based on this report.  It is 

17    Defendant's Exhibit 516.  

18               MR. COHEN:  I am told this document is not 

19    relevant, your Honor.  

20               MR. BARON:  I think it is highly relevant.  

21               MR. ZIMROTH:  Doesn't that give you a lot of 

22    guidance, your Honor?  

23               THE COURT:  Can we smooth the numbers? 

24               (Laughter)

25               I will take it.  516 is admitted.
                                                              2330

 1               (Defendant's Exhibit 516 marked for 

 2    identification was received in evidence.) 

 3               MR. COHEN:  Your Honor, there are also a number 

 4    of deposition designations.  We can take that up at a later 

 5    time.  

 6               THE COURT:  Not now, some other time.

 7               Let's break for lunch.  We will resume with the 

 8    last witness, I believe.  

 9               MR. MILLET:  That's correct, your Honor.  

10               THE COURT:  We will resume at 2:00 o'clock.

11               (Luncheon recess) 

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2331

 1                         AFTERNOON SESSION
                           _________________

 2                             2:00 p.m.

 3               THE COURT:  Mr. Sitcov, we missed you.  

 4               MR. SITCOV:  I missed you.  

 5               THE COURT:  Things have been rational.  

 6               MR. SITCOV:  I can fix it.  
                                             

 7               THE COURT:  I take it you have one more witness?  

 8               MR. SITCOV:  Yes, we do.  The defendants call 

 9    David Freedman.  

10    DAVID FREEDMAN, 
                      

11         called as a witness by the defendants, having

12         been duly sworn, testified as follows: 

13               THE COURT:  Would you state your full name and 

14    spell your last name for the record.  

15               THE WITNESS:  My name is David Freedman, F R E E 

16    D M A N.  

17    DIRECT EXAMINATION 

18    BY MR. SITCOV:

19         Q.    Dr. Freedman, where are you employed?

20         A.    I am employed at the University of 

21    California-Berkeley.

22         Q.    What is your occupation?

23         A.    I am a professor of statistics.

24         Q.    Would you open the black binder at your left and 

25    turn to Defendants' Exhibit 61.  Dr. Freedman, is that a 
                                                              2332

 1    copy of your curriculum vitae?

 2         A.    Yes.  

 3               MR. SITCOV:  Defendants move the admission of 

 4    Defendants' Exhibit 61.  

 5               MR. SOLOMON:  No objection, your Honor.  

 6               THE COURT:  61 is admitted

 7               (Defendants' Exhibit 61 for identification was 

 8    received in evidence)

 9         Q.    Can you briefly summarize your post-high school 

10    education, Dr. Freedman?

11         A.    Yes.  I got a Bachelor's degree at McGill 

12    University in Montreal, and a Master's degree and Ph.D. at 

13    Princeton.

14         Q.    What field was your Ph.D in at Princeton?

15         A.    Probability and statistics.

16         Q.    How long have you been employed at Berkeley?

17         A.    Since 1961.

18         Q.    Are you a full professor of statistics?

19         A.    I am.

20         Q.    Have you published any books in the field of 

21    statistics?

22         A.    Yes.

23         Q.    How many?

24         A.    Five.

25         Q.    Would you turn to page 9 of your CV.  Are those 
                                                              2333

 1    the books that you are referring to?

 2         A.    Yes.

 3         Q.    Are any of them textbooks?

 4         A.    I think they all are.

 5         Q.    Are any of them used at universities other than 

 6    Berkeley?

 7         A.    Yes, they are, especially number 4.

 8         Q.    About how many universities other than Berkeley 

 9    is that used at, do you know?

10         A.    I think several hundred, all over the United 

11    States, Canada, Australia.

12         Q.    Have you ever published articles in the field of 

13    statistics in any scientific journals?

14         A.    Yes.

15         Q.    Approximately how many have you published?

16         A.    About a hundred.

17         Q.    Could you turn to the bibliography section of 

18    your CV.

19         A.    Yes.

20         Q.    Does that reflect the publications you just 

21    discussed?

22         A.    Yes.

23         Q.    Do any of those publications concern the issue of 

24    a statistical adjustment of the decennial census?

25         A.    Yes.
                                                              2334

 1         Q.    How many concern that topic?

 2         A.    There are two published and one in press.

 3         Q.    Have you ever lectured in the field of statistics 

 4    outside of your classroom duties at Berkeley?

 5         A.    Yes.

 6         Q.    How many times?

 7         A.    At a rough estimate, upwards of a hundred.

 8         Q.    Have you ever given expert testimony before the 

 9    United States Congress on the issue of a statistical 

10    adjustment of the decennial census?

11         A.    Yes.

12         Q.    How many times?

13         A.    Twice in person and once written.

14         Q.    Have you ever been qualified in federal court as 

15    an expert on the subject matter of statistics?

16         A.    Yes.

17         Q.    Have you ever been qualified in federal court as 

18    an expert on the issue of census adjustment?

19         A.    Yes.  

20               MR. SITCOV:  Your Honor, at this time defendants 

21    offer David Freedman as an expert in statistics.  

22               MR. SOLOMON:  Your Honor, I will be brief.  Your 

23    Honor may have ruled on some of these matters before.  I 

24    think this is a particularly egregious case I would like to 

25    call to your attention.  So long as this witness confines 
                                                              2335

 1    his opinions to what was discussed at the deposition, we 

 2    don't per se object to his offering opinions.  We object to 

 3    the testimony on three grounds.  I will be brief.  

 4               The testimony relates to all post-July 15th 

 5    materials.  

 6               THE COURT:  Post-July 15th?  

 7               MR. SOLOMON:  Materials, data, analysis.  For the 

 8    reasons that your Honor excluded evidence that the 

 9    plaintiffs tried to proffer because it does not go to the 

10    state of mind of the Secretary or the Bureau, it should not 

11    be admitted here. 

12               Second, there is an additional reason here.  

13    Professor Freedman changes the analyses of the Bureau, 

14    doesn't use the data of the Bureau, has run his own data, 

15    has run his own models.  So we are not even in a case -- for 

16    example, Dr. Wachter, as a member of the Special Advisory 

17    Panel, maybe could explicate what the Bureau was doing.  But 

18    the purpose of this testimony is quite the contrary.  

19               Third and most significantly, we have been asking 

20    for months for the data underlying this witness's opinions 

21    and testimony.  We know that his reports long predated the 

22    date of his deposition.  We weren't given the reports.  We 

23    weren't given the data.  

24               Even now we have not been given any data that 

25    Professor Freedman has created.  And I am prepared to show 
                                                              2336

 1    your Honor that he has created his own data.  It is not 

 2    Bureau data that he is going to talk to you about.  

 3               For those reasons, we think that it is impossible 

 4    to cross-examine him.  

 5               I do note we were given yesterday for the first 

 6    time three computer disks.  Two of the disks relate to 

 7    Professor Wachter.  Put them aside.  They relate to P-12.  

 8    This witness isn't testifying to the P studies, he told us 

 9    at his deposition, so we put that aside.  

10               The third disk relates to variances and 

11    covariances.  That is related to what he is doing, but the 

12    data appears to be what the Bureau generated, not what this 

13    witness generated.  We also haven't been given any programs.  

14    We cannot use what we have been given.  It is therefore 

15    impossible for us fairly to cross-examine him.  

16               For those three reasons, we would object.  

17               THE COURT:  Mr. Sitcov, would you take them in 

18    reverse order.  First, the data underlying his opinions. 

19               MR. SITCOV:  We gave them all the data that Dr. 

20    Freedman used, plain and simple.  

21               THE COURT:  He says otherwise.  

22               MR. SITCOV:  He is wrong.  

23               THE COURT:  You are wrong, Mr. Solomon.  

24               MR. SOLOMON:  May I have voir dire of this 

25    witness?  I think we will demonstrate that.  
                                                              2337

 1               THE COURT:  No smile even on that?  

 2               MR. SOLOMON:  Especially on that.  

 3               THE COURT:  Go ahead.  

 4    VOIR DIRE EXAMINATION 

 5    BY MR. SOLOMON:

 6         Q.    Dr. Freedman, are you aware of the disks that 

 7    were handed to us yesterday?

 8         A.    Sure.

 9         Q.    Did you prepare those disks for delivery to us?

10         A.    I did.

11         Q.    Two of the disks involve P-12 data, is that 

12    right?

13         A.    They do.

14         Q.    The third disk involves variance/covariance 

15    matrices, has variance/covariance matrices on them, is that 

16    right?

17         A.    Among other things.

18         Q.    The variance/covariance matrices that are on 

19    there, are they the matrices of the Bureau?

20         A.    Yes.

21         Q.    In the work that you did and the opinions that 

22    you are intending to offer, you didn't use those covariance 

23    matrices; you created your own, correct?

24         A.    Well, let's be clear.  I created my own using the 

25    ingredients that are in other files on those disks that you 
                                                              2338

 1    did not mention.

 2         Q.    Disks that we did not mention?

 3         A.    Well, I misspoke then.  I created 

 4    variance/covariance matrices using ingredients in data files 

 5    on disk C that was provided to you that you did not mention 

 6    in a prior question. 

 7         Q.    Have you given us the programs that you used to 

 8    run the data?

 9         A.    No, I have not.

10         Q.    How long, in your estimation, since my computer 

11    people tell me it would take weeks, how long in your honest 

12    estimation would it take someone who got the disk, the third 

13    disk that you gave us, to recreate and rerun your programs?

14         A.    Several weeks to several months 

15               MR. SOLOMON:  Your Honor, we cannot do it.  We 

16    have not been able to do it.  We cannot test the witness's 

17    opinions.  We have no idea, in fact, what the basis of his 

18    testimony is because it is, in fact, different from the 

19    Bureau's.  

20               THE COURT:  Was Dr. Freedman subject to a 

21    deposition?  

22               MR. SOLOMON:  He was, your Honor.  

23               THE COURT:  Was he deposed?  

24               MR. SOLOMON:  He was.  

25               THE COURT:  Why isn't that adequate?  
                                                              2339

 1               MR. SOLOMON:  At the deposition Dr. Freedman had 

 2    already done the analysis, but we were prohibited from 

 3    asking Dr. Freedman to discuss anything that represented a 

 4    communication or work arising out of a communication between 

 5    himself and the government.  The government asserted a work 

 6    product objection.  

 7               Since we didn't have his reports at the time of 

 8    the deposition, we couldn't go through the reports and say 

 9    what is the basis for this, what is the basis for this.  So 

10    the deposition was, although ably taken -- I did not take it 

11    -- the people who took the deposition did the best they 

12    could.  In the deposition, your Honor, we couldn't even get 

13    what his opinion was because we didn't have the report, much 

14    less the data that he relies on.  

15               I have to make it clear, your Honor.  The data is 

16    technical, reams and reams, millions of bytes of data, that 

17    unless you know how it is put together, unless you know the 

18    program it is run on, you don't have any idea.  The witness 

19    just said weeks or months to figure it out.  

20               MR. SITCOV:  Your Honor, we gave them the facts 

21    or data that he used just as Rule 705 requires.  They were 

22    certainly free to ask Dr. Freedman his opinions, and they 

23    did for two days.  

24               THE COURT:  Did not?  

25               MR. SITCOV:  Did.  They asked him for two days.  
                                                              2340

 1    They asked him all about his computations, and he told them 

 2    all about his computations.  He is simply using the Bureau's 

 3    data, and he used the Bureau's data to form his opinion that 

 4    he is going to testify about and that the plaintiffs deposed 

 5    him about for two days.  

 6               MR. SOLOMON:  There is a very important factual 

 7    predicate to your Honor's ability to rule on that:  That I 

 8    be permitted to establish that the data is not the Bureau's 

 9    data.  The data could have been taken from Bureau data, that 

10    we don't know.  That is going to take us two or three months 

11    to figure out.  

12               But then the data was changed.  In the reports 

13    that we received on May 1 there are statements made and 

14    computations right there on the page.  But it is not Bureau 

15    data.  It is Professor Freedman's own data.  We did ask 

16    immediately at the deposition and immediately prior for this 

17    data.  On May 1 when we received the reports, we asked in 

18    writing and we were refused.  We have asked twice since in 

19    writing and have been refused.  

20               Particularly in light of the fact that he is not 

21    coming to explicate what the Bureau did, but he is coming to 

22    give his own analysis and I don't have any idea what the 

23    data is underlying that analysis, his testimony should be 

24    precluded.  

25               MR. SITCOV:  First of all, your Honor -- 
                                                              2341

 1               THE COURT:  May I interrupt?  Mr. Sitcov?  

 2               MR. SITCOV:  First of all, I remind the Court 

 3    that the plaintiffs made this exact argument in advance of 

 4    trial, and the Court denied plaintiffs' motion to exclude 

 5    this evidence on the same grounds.  

 6               THE COURT:  Exclude?  

 7               MR. SITCOV:  Exclude this type of evidence on the 

 8    same grounds.  

 9               THE COURT:  I don't recall this.  With respect to 

10    Dr. Freedman?  

11               MR. SITCOV:  The plaintiffs wanted you to exclude 

12    any analyses that were done after the date of the 

13    Secretary's decision.  

14               THE COURT:  Oh, all right.  I'm with you.  

15               MR. SITCOV:  They made this exact argument at 

16    that time, and the Court rejected it.  

17               Secondly, Dr. Freedman's data is the Bureau's 

18    data.  

19               THE COURT:  Mr. Solomon keeps saying it isn't.  

20    You say it is.  

21               MR. SITCOV:  Is your data the Bureau's data?  

22               THE WITNESS:  As far as I can tell, every single 

23    number in my report was computed from data provided to me by 

24    the Bureau.  So the numbers in my report represent the 

25    results of analysis I had done starting from Bureau data.  
                                                              2342

 1               Now, my analysis is different from the Bureau's, 

 2    of course it is.  But it is their data.  I didn't go and run 

 3    my own PES.  

 4               THE COURT:  You are going on your own?  

 5               THE WITNESS:  I am going on my own, but I 

 6    certainly didn't run my own post-enumeration survey and 

 7    collect field data and stuff like that.  

 8               MR. SOLOMON:  Your Honor, may I be heard?  

 9               THE COURT:  Yes.  

10               MR. SOLOMON:  The Bureau did smoothing and 

11    presmoothing.  It has an effect, we believe it is a small 

12    effect, that is part of the issue here, it has an effect on 

13    the numbers.  

14               Dr. Freedman took data.  We don't know what data 

15    because as he has just admitted we can't figure out what 

16    data he used because we didn't get the programs.  He took 

17    some data.  

18               THE COURT:  But it is all Bureau data.  

19               MR. SOLOMON:  He took some data and then 

20    manipulated the data.  He changed the data.  Because while 

21    the Bureau did smoothing and resmoothing, the Professor 

22    sometimes didn't presmooth, sometimes he did presmooth.  

23               There are charts in his papers, poststratum 49 -- 

24               THE COURT:  Wouldn't all that affect the weight 

25    of the testimony only?  
                                                              2343

 1               MR. SOLOMON:  Your Honor, I concede to you, and I 

 2    learned this from your treatise, in all but an extreme case 

 3    it should go to the weight and it should go to our ability 

 4    to cross-examine.  This I maintain is a very extreme case 

 5    because all of the data that we are talking about we have to 

 6    guess at.  We don't have the data and we don't have the 

 7    programs.  

 8               With respect to the motion that Mr. Sitcov was 

 9    talking about, I know your Honor has already heard the 

10    argument.  I am not going to belabor it.  The motion 

11    addressed to information and studies of the Bureau.  I can 

12    understand that your Honor would be interested, might be 

13    interested -- we don't agree, but I can understand you might 

14    be interested -- in what the Bureau has been doing.  But not 

15    about what this witness has been doing.  

16               I am also prepared to show to your Honor that the 

17    Bureau has on at least a couple of occasions rejected the 

18    analysis of this witness.  So this witness comes in, 

19    notwithstanding the fact that the Bureau has rejected his 

20    analysis, he makes his analysis anyway.  We don't know what 

21    the data is.  

22               I found a better quote in Weinstein than in 

23    yours, but it is the same all over. 

24               THE COURT:  I must have written it for him.   

25               MR. SOLOMON:  "Data produced by a process or 
                                                              2344

 1    system such as computers or surveys must be made known to 

 2    opponents in courts well in advance of trial so that the 

 3    accuracy of the system and its results can be checked by 

 4    experts if necessary."  

 5               We have been completely deprived of the ability 

 6    to do that.  

 7               MR. SITCOV:  First of all, your Honor, they have 

 8    had the data.  Whether Weinstein said it or you said it, 

 9    they got what both of you want.  

10               Secondly, Dr. Freedman's papers, both of Dr. 

11    Freedman's papers, have exhaustive technical descriptions of 

12    exactly what Dr. Freedman did with the data.  

13               THE COURT:  Both of his papers?  What are you 

14    referring to?  

15               MR. SITCOV:  As you will see when he testifies, 

16    he has prepared three reports based on his work.  Those 

17    reports provide detailed descriptions of what he did with 

18    the data, and they identify the data he used.  

19               THE COURT:  When was that given to the 

20    plaintiffs?  

21               MR. SITCOV:  That was given to the plaintiffs May 

22    1st.  

23               THE COURT:  I understand your point, Mr. Solomon.  

24    It seems to me that rather than to exclude Dr. Freedman's 

25    testimony entirely, the more prudent course, to adopt an 
                                                              2345

 1    adjective our President loves, would be to hear his 

 2    testimony, then to hear your, I'm sure, blistering 

 3    cross-examination, and to give the lack of an opportunity to 

 4    have prepared all the blisters you would like appropriate 

 5    probative weight in evaluating the testimony.  

 6               I will therefore overrule the objection and 

 7    permit Mr. Sitcov to continue.  

 8               Where were you?  You had offered the resume?  

 9               MR. SITCOV:  Yes.  Oh, no.  You had accepted it 

10    and he is now, I take it, qualified to offer an opinion?  

11               THE COURT:  Yes.  He is qualified to express his 

12    opinion on statistical matters.  

13    BY MR. SITCOV:

14         Q.    Dr. Freedman, have you ever testified in courts 

15    as an expert against the United States?

16         A.    Yes.

17         Q.    When was that?

18         A.    A couple of years ago.

19         Q.    Who represented the United States in that case?

20         A.    Department of Justice.

21         Q.    Have you ever testified in court as an expert on 

22    behalf of any plaintiff in this case?

23         A.    Yes.  Actually, it was the same case.  I 

24    testified for the County of Los Angeles.

25         Q.    When did you first agree to be retained as an 
                                                              2346

 1    expert by the defendants in this case?

 2         A.    Spring 1989.

 3         Q.    Are you being paid for your services as an expert 

 4    in this case?

 5         A.    I am.

 6         Q.    How much are you being paid?

 7         A.    I am being paid $300 an hour plus expenses.

 8         Q.    How many hours have you billed for?

 9         A.    At a rough estimate, upwards of 500.

10         Q.    Dr. Freedman, do you think there was an 

11    undercount in the 1990 Decennial Census?

12         A.    Yes, sir.

13         Q.    Do you think the undercount is differential?

14         A.    Yes.

15         Q.    Do you think the 1990 Decennial Census should be 

16    adjusted to correct for the differential undercount?

17         A.    No, I do not.  I don't think that that is really 

18    possible.

19               THE COURT:  Possible?  

20               THE WITNESS:  Possible.

21         A.    It is possible to do the arithmetic, but I don't 

22    think it is possible to do it in a way which gets you closer 

23    to the truth.

24         Q.    Why is that?

25         A.    I think there are three main problem areas with 
                                                              2347

 1    adjustment:  Sampling error, which I expect to be testifying 

 2    about; nonsampling error, which I am sure will come up; and 

 3    failures in the homogeneity assumption, which I believe Dr. 

 4    Wachter discussed earlier.

 5         Q.    Does the adjustment sought by the plaintiffs in 

 6    this case involve smoothing?

 7         A.    It does.

 8         Q.    What is the objective of smoothing?

 9         A.    The objective is to reduce sampling error in the 

10    estimated adjustments.

11         Q.    In your opinion, does smoothing solve the 

12    problems created by a sampling error?

13         A.    I don't think it does.

14         Q.    Have you considered whether loss function 

15    analysis supports a decision to adjust the 1990 Decennial 

16    Census?

17         A.    I have.

18         Q.    In your opinion, does loss function analysis 

19    support a decision to adjust the 1990 Decennial Census?

20         A.    No, I don't believe it does.

21         Q.    Did you prepare any reports that reflect your 

22    opinion about smoothing?

23         A.    I did.

24         Q.    Have you prepared any reports about whether loss 

25    function analysis supports a decision to adjust the 1990 
                                                              2348

 1    Decennial Census?

 2         A.    I did.

 3         Q.    Could you turn to Exhibits 62, 63, and 64 in the 

 4    binder in front of you.

 5         A.    Yes.

 6         Q.    Can you identify those documents?

 7         A.    62 and 63 are reports on smoothing.  64 is a 

 8    report on loss functions.

 9         Q.    Did you write Exhibit 62 yourself?

10         A.    I did.

11         Q.    Did you do the data analysis yourself?

12         A.    Most of it, yes.

13         Q.    Did you do the calculations yourself?

14         A.    Most of them.

15         Q.    Did you write Defendants' Exhibit 63 yourself?

16         A.    No.  63 was written by Dan Koster and Richard 

17    Cutler, although I had considerable input into that.

18         Q.    Did you do the data analyses in Defendants' 

19    Exhibit 63?

20         A.    I did some of it, but most of was done by Koster 

21    and Cutler.

22         Q.    Did you do the calculations in Defendants' 

23    Exhibit 63?

24         A.    Most of the work was done by Koster and Cutler.  

25    I did checks on the calculations.
                                                              2349

 1         Q.    Did you write Defendants' Exhibit 64?

 2         A.    Yes

 3         Q.    Did you do the date a analysis that is included 

 4    in 64 yourself?

 5         A.    Yes.

 6         Q.    Did you do the calculations yourself?

 7         A.    Most of them, yes.  

 8               MR. SITCOV:  Defendants move 62, 63, and 64, move 

 9    they be admitted into evidence, your Honor.  

10               MR. SOLOMON:  Your Honor, we object to each and 

11    every one of them.  I understand your Honor's ruling that 

12    this witness's opinions will be taken -- that your Honor 

13    will take into account that we have not had any of the data.  

14    But to allow in this hearsay which is full of not only 

15    single hearsay, there is double hearsay, there is triple 

16    hearsay, I can't cross-examine him, blistering or not.  It 

17    will take a week to crawl through all of this.  

18               I don't believe this is going to help your Honor 

19    at all figure out what is going on here.  This witness is 

20    capable of offering his opinions.  Particularly in light of 

21    the fact that we don't have the data underlying this, 

22    particularly in light of the fact that I cannot 

23    cross-examine, by the way, your Honor, some of the hearsay 

24    statements that are in here because they are hearsay 

25    statements about Bureau people, including Director Bryant -- 
                                                              2350

 1    I can't call her.  I tried.  I would love for the defendants 

 2    to call her.  I cannot cross-examine those.  

 3               There are materials in here that I believe are 

 4    incompetent for this expert to rely on.  In particular, 

 5    there are references to Dr. Breiman, which your Honor has 

 6    already found is not reasonably relied upon by an expert.  

 7    To so move in this mountain of paper full of detail I 

 8    believe is objectionable.  

 9               THE COURT:  I don't quite understand, Mr. Sitcov.  

10    What is the purpose of admitting this, to borrow Mr. 

11    Solomon's phrase, mountain of paper?  

12               MR. SITCOV:  It is more of a molehill as compared 

13    to some of the stuff we have seen.  But the purpose is, your 

14    Honor, that it is a clear and concise explanation of Dr. 

15    Freedman's examination of the data he considered and of the 

16    opinions that he has developed on the basis of that data.  

17    It is simply not true, as I said before, that the plaintiffs 

18    don't have the data.  They do.  

19               THE COURT:  This is it?  

20               MR. SITCOV:  No.  The data is in the data files 

21    that Dr. Freedman discussed before, the computer files.  But 

22    they do have the data.  And all of the conclusions in there 

23    are sourced in the back of each report.  

24               THE COURT:  Of?  

25               MR. SITCOV:  Of the reports, yes.  For 62 and 64 
                                                              2351

 1    there are extensive technical appendices which describe in 

 2    detail not only the data used but the manner in which it was 

 3    used.  Certainly, like any other expert, Dr. Freedman is 

 4    free to rely on the work of others in the field of his 

 5    expertise in order to develop an opinion.  

 6               THE COURT:  If I understand you, and I am not at 

 7    all sure I do, Mr. Sitcov -- I have learned to live with 

 8    that -- if I understand what you are telling me, Dr. 

 9    Freedman will be expressing his opinions here today.  Those 

10    opinions are based on what we will hear from him today, also 

11    on Exhibits 62, 63, and 64.  

12               The latter three documents, in turn, are resting 

13    on data which you say Mr. Solomon has and which he says -- 

14    first he said he didn't have it, then he said he does have 

15    it but he can't possibly read it for three months.  

16               MR. SITCOV:  Yes, I think Mr. Solomon said he has 

17    it but can't read it for three months.  

18               THE COURT:  Have I analyzed where you are coming 

19    from correctly?   

20               MR. SITCOV:  Not entirely.  

21               THE COURT:  Then I don't understand you.  

22               MR. SITCOV:  These reports simply reflect Dr. 

23    Freedman's opinions.  

24               THE COURT:  Which we are going to hear here 

25    today?  
                                                              2352

 1               MR. SITCOV:  Yes.  

 2               THE COURT:  So why do I need the report to say it 

 3    twice?  

 4               MR. SITCOV:  The reports probably say it somewhat 

 5    more fully and in perhaps a fashion that may assist the 

 6    Court later in its deliberations.  

 7               THE COURT:  Tell you what.  Let's leave them out 

 8    for the moment.  Let me hear the testimony.  If you wish, 

 9    you can renew your offer later on, at which point I will be 

10    in a much more intelligent position to evaluate all this.  

11               MR. SITCOV:  Fine.  

12    BY MR. SITCOV:

13         Q.    Dr. Freedman, last year at this time did you have 

14    an opinion about whether the 1990 Decennial Census should be 

15    adjusted?

16         A.    Yes.

17         Q.    What was your opinion about adjustment then?

18         A.    I didn't think it was going to work.

19         Q.    In 1987 did you have an opinion about -- 

20               THE COURT:  Last year at this time.  Where were 

21    we in the scheme of things?  

22               MR. SITCOV:  Just before the decision.  The 

23    decision was July 15th.  So we would have been about if ten 

24    weeks before.  

25               THE COURT:  All right.
                                                              2353

 1         Q.    In 1987 did you have an opinion about whether the 

 2    1990 Decennial Census should be adjusted?

 3         A.    Yes, sir, I did.

 4         Q.    What was your opinion then?

 5         A.    Essentially, the same.  I did not see much 

 6    prospect of making a successful adjustment.

 7         Q.    Did you do anything to make your opinion known to 

 8    officials at the commerce department or the Census Bureau in 

 9    1987?

10         A.    Yes, I did.

11         Q.    What did you do?

12         A.    I went to the Bureau.  I spent a day there 

13    talking to them to find out what their procedures were, how 

14    they were different from 1908.  This is really before I had 

15    formed much of an opinion.  I talked to them about their 

16    procedures.  

17               They sent me a ton of documents, which just 

18    dwarfed this binder.  I went through them.  I had quite an 

19    expensive series of exchanges by letter with Kirk Wolter.  I 

20    had numerous telephone calls with Kirk.  

21               It seems to me I testified in Congress at a 

22    hearing of Chairman Dymalley's committee, which was a joint 

23    event with the American Statistical Association.  There were 

24    many people from the Bureau present at that hearing, 

25    including Dr. Wolter.  He certainly had a copy of my 
                                                              2354

 1    testimony before the hearing, and he did comment on it.  

 2               It seems to me I had some considerable exchange 

 3    with Bureau people over those issues at around that time the 

 4    way I have just described them.

 5         Q.    Why did you make your opinion known to the Census 

 6    Bureau then?

 7         A.    Partly, I thought there was some hope of talking 

 8    them out of a program which I did not think was likely to 

 9    work.  Partly, I did not want to disagree with them in 

10    public and I wanted to keep the issues that I was 

11    disagreeing with them about as narrow as possible.

12         Q.    Was Barbara Bailar employed at the Census Bureau 

13    during this time?

14         A.    Yes, she was.  In fact, when I went to visit the 

15    Bureau that day, she was there for a large part of the day.

16         Q.    Do you have an opinion about whether the 1980 

17    decennial census should have been adjusted?

18         A.    I do.

19         Q.    What is your opinion?

20         A.    I testified against it in 1984.  I did not think 

21    that adjustment was likely to improve the accuracy of the 

22    census.  And that remains my position today.

23         Q.    Is your opinion on adjustment based on any Census 

24    Bureau or commerce department documents?

25         A.    In part, yes.
                                                              2355

 1         Q.    What are those documents?

 2         A.    I am thinking mainly of the Secretary's decision 

 3    document that came out in all those volumes, the P studies, 

 4    a set of documents that I got from the Bureau describing the 

 5    smoothing model.  I think those would be the main documents.

 6         Q.    In your opinion, do those documents consider the 

 7    areas you think need to be considered in deciding whether to 

 8    adjust the 1990 Decennial Census?

 9         A.    Yes, I believe they do.

10         Q.    What are those areas?

11         A.    Those documents address the question of sampling 

12    error, they address nonsampling error, they address 

13    heterogeneity, they address measures of accuracy for the 

14    census, for the adjustment.  They address differential 

15    undercount.  I think those are principal issues.  

16               Some of those issues are addressed better than 

17    others, but they are in there.  Smoothing is discussed.  I 

18    would see those documents as addressing the main questions.

19         Q.    Have you reviewed Secretary Mosbacher's July 15, 

20    1991 decision not to adjust the decennial census?

21         A.    Yes.

22         Q.    In your opinion, did he consider the relevant 

23    questions on the issue of adjustment?

24         A.    Yes, I believe he did.  In general, those are the 

25    ones I was just mentioning.  He discussed the undercount in 
                                                              2356

 1    the census, the differential undercount, sampling error, 

 2    nonsampling error, heterogeneity, loss function analysis 

 3    tests.  

 4               He raised, and I think properly settled, some 

 5    critical questions about whether you want to, if you have to 

 6    make a choice, whether you want to focus on the accuracy of 

 7    the numbers, numeric accuracy, or the accuracy of shares, 

 8    distributive accuracy.  He focused on the question of 

 9    whether you want to, if you have to make a choice, focused 

10    on accuracy of results for groups or for places.  He chose 

11    places.  

12               He decided what kind of evidence he wanted before 

13    he decided to make the adjustment.  Those seemed to me to be 

14    the critical questions, and he addressed them.

15         Q.    In your opinion, did Secretary Mosbacher consider 

16    those questions in a rational way?

17         A.    Yes, I believe he did.  I mean, I think he made 

18    some mistakes, as anyone would.  But I think his 

19    considerations were rational.

20         Q.    Have you been in court during the plaintiffs' 

21    presentation of the case?

22         A.    I have been here for much of their case, and I 

23    have read the transcript for those days when I was not here.

24         Q.    In your opinion, does any of the testimony that 

25    the plaintiffs offered make a strong argument for 
                                                              2357

 1    adjustment?  

 2               MR. SOLOMON:  Objection.  

 3               THE COURT:  Sustained.

 4         Q.    You mentioned, Dr. Freedman, that the Secretary's 

 5    July 15th decision contained some mistakes?

 6         A.    Yes.

 7         Q.    Can you think of any offhand?

 8         A.    One that comes to mind is he referred to an 

 9    analysis based on an absolute error loss function when 

10    apparently he meant squared error, at least from context.  

11               Another example is his discussion of the 

12    so-called fourth cell in the dual-system estimator.  His 

13    estimate of the number of people in the fourth cell seems 

14    high to me, although I have not calculated the correct 

15    number myself, and I don't follow his discussion of the sex 

16    ratio in that cell.

17         Q.    Do any of those mistakes suggest to you that the 

18    Secretary should have decided in favor of adjustment?  

19               MR. SOLOMON:  Objection.  

20               THE COURT:  Overruled.  You may answer.

21         A.    No, I don't believe they do.

22         Q.    You will see at your right hand, Dr. Freedman, 

23    there is a copy of what is Plaintiffs' Exhibit 9.  It is the 

24    Secretary's decision.  Would you turn to page 2-2.

25         A.    Yes.
                                                              2358

 1         Q.    In the paragraph that carries over from the 

 2    previous page, there is a sentence, the last sentence.  

 3    There the Secretary says, "There is certainly not sufficient 

 4    evidence to reject the distributive accuracy of the census 

 5    counts in favor of the adjusted counts."  

 6               Do you agree with that statement of the 

 7    Secretary's?

 8         A.    Yes, I do.

 9         Q.    Would you turn to page 2-3, the second full 

10    paragraph on that page says, "One of the most problematic 

11    parts of the adjusted process was the bundle of statistical 

12    techniques contained in the smoothing process.  These 

13    techniques relied heavily on statistical assumptions, 

14    resulted in large changes in adjustment factors, and may 

15    very well have led to an overstatement of the undercount."  

16               Do you agree with that statement?

17         A.    Yes, I do.  I know that John Rolph, who is 

18    sitting here in court, specifically criticized that 

19    paragraph.  Much as I like Dr. Rolph and enjoy his company, 

20    I am just going to have to differ with his opinion in this 

21    matter.

22         Q.    Dr. Freedman, what does "robust" mean as applied 

23    to a statistical procedure?

24         A.    I am not sure it has a sharp technical meaning, 

25    but in general what it means is that if you change the 
                                                              2359

 1    details of the procedure or change the assumptions, you will 

 2    not see much of a change in the outputs from the procedure.

 3         Q.    In your opinion, was the smoothing process used 

 4    to produce the adjusted counts robust?

 5         A.    No, I don't believe it was.

 6         Q.    In your opinion, was the loss function analysis 

 7    that the Census Bureau conducted robust?

 8         A.    No, I don't believe that was either.  I know 

 9    there is contrary opinion from Dr. Fisher, but I must tell 

10    you my views, and my views are as I stated them.

11         Q.    Dr. Freedman, could you turn to Defendants' 

12    Exhibit 72.  Do you recognize the chart on that page?

13         A.    Yes.

14         Q.    What does that chart show?

15         A.    It tries to indicate in a schematic way two of 

16    the choices that the Secretary had to make in deciding about 

17    adjustment.  You can, sometimes these are consistent, 

18    sometimes these are inconsistent.  You can try to make the 

19    numbers of people better by adjusting the census, or you can 

20    try to make the shares better.  

21               The columns indicate another choice.  You can 

22    focus on subgroups of the population, for example, ethnic or 

23    racial groups, or you can focus on geographical places, 

24    states or cities or counties.  

25               The little tick mark indicates the Secretary's 
                                                              2360

 1    choice, as I read the document, namely, we want to try and 

 2    make shares better rather than numbers -- that is 

 3    distributive accuracy rather than numeric accuracy in the 

 4    jargon -- and we want to try and make shares better for 

 5    places -- states, cities, counties -- rather than for 

 6    groups.  

 7               Sometimes you can do everything, but sometimes 

 8    there is a conflict.  In case of conflict, the tick mark 

 9    indicates the resolution in the guidelines, as I understand 

10    them.

11         Q.    In your opinion, was the choice the Secretary 

12    made a reasonable one?  

13               MR. SOLOMON:  Objection.  

14               THE COURT:  Overruled.

15         A.    I would say more than reasonable.  I believe he 

16    made the right choice in these two areas.

17         Q.    Why do you believe it to be the right choice?

18         A.    I think a large part of the argument about 

19    adjusting the census is because the census in our country -- 

20    I hope I may speak that way.  I am only a resident alien, 

21    but I still think of it as our country -- are sources are 

22    allocated to places -- states, cities, counties, towns.  Tax 

23    monies, Congressional seats, are not allocated to groups.  

24    That is why I think the right choice in case of a conflict 

25    is for accuracy at the level of geographical areas.  
                                                              2361

 1               In terms of the choice between numbers and 

 2    shares, I have heard plaintiffs argue that there are some 

 3    grant programs which depend on absolute numbers.  There are 

 4    some thresholds.  For example, if your town has over 100,000 

 5    people, you might be eligible for one kind of grant.  If 

 6    your town has less than 100,000 people, you might not be 

 7    eligible.  I believe that to be true.  

 8               Still and all, what census adjustment is about in 

 9    terms of resources is reallocating those resources.  It is a 

10    different way of sharing the pie.  Adjusting the census does 

11    not make new tax money, does not make more seats in 

12    Congress.  If there is a winner, there has to be a loser.  

13               For that reason, I think shares are more 

14    important than numbers, distributive accuracy is more 

15    important than numeric accuracy.  You would like to have 

16    both, but in case of conflict I think shares are more 

17    important than numbers.

18         Q.    Dr. Freedman, could you turn back again to the 

19    secretaries's decision, that is, Plaintiffs' Exhibit 9.  I 

20    will ask you to turn to page 2-26.  

21               Would you look, please, at the first full 

22    paragraph on that page, the one that begins "Choice of 

23    Accuracy."  I want you to consider the second sentence in 

24    that paragraph, which reads, "Simply correcting for the 

25    estimated net undercount can improve numeric accuracy but 
                                                              2362

 1    significantly worsen distributive accuracy. " 

 2               In your opinion, is that a problem that should be 

 3    considered before a decision is made to adjust the census?

 4         A.    Yes, I think so.

 5         Q.    Can you explain how that problem the Secretary 

 6    was concerned about could occur?

 7         A.    You might have a situation in which there is a 

 8    net undercount in every single state.  If you add people to 

 9    every state, if you don't go too far, you will be improving 

10    the numeric accuracy of the census by that adjustment.  

11               However, if you put people in the wrong state, 

12    you could be harming the distributive accuracy.  In other 

13    words, there are situations in which you can improve numeric 

14    accuracy but worsen distributive accuracy.

15         Q.    Do you consider that a likely event should the 

16    Court order the adjustment that the plaintiffs seek?  

17               MR. SOLOMON:  Objection.  

18               THE COURT:  I don't understand.  Does he consider 

19    what likely? 

20               MR. SITCOV:  The problem he was just describing.  

21    He just described the problem where you could improve 

22    numeric accuracy while degrading distributive accuracy.  

23    What I want to know is if, in his opinion, he considers that 

24    that is something that is likely to occur in the adjustment 

25    that the plaintiffs seek is ordered by the Court.  
                                                              2363

 1               THE COURT:  To improve the one and degrade the 

 2    other?  

 3               MR. SITCOV:  Yes.  

 4               THE COURT:  That it is to be a likely result?  

 5               MR. SITCOV:  Yes.  

 6               THE COURT:  All right.  The objection is 

 7    overruled.  You may answer.

 8         A.    I think that is a serious possibility, yes.  I 

 9    think the adjustment that the Bureau was considering could 

10    easily improve numeric accuracy.  I have not made any 

11    studies of that.  I am not offering that as my own opinion, 

12    but it seems reasonable to me as an outcome.  

13               I think there is a real chance that the 

14    adjustment would degrade the accuracy of shares, the 

15    distributive accuracy.

16         Q.    In your opinion, could the adjustment that the 

17    plaintiffs seek make the shares of the population better for 

18    racial or ethnic groups, while making the shares worse for 

19    many, if not most, states?  

20               THE COURT:  Most?  

21               MR. SITCOV:  States.  

22               MR. SOLOMON:  Your Honor, I object here because 

23    the question calls for speculation, and form.  

24               THE COURT:  Overruled.  You may answer.  

25         A.    Yes.  Again, I believe that to be distinctly 
                                                              2364

 1    possible if the census is adjusted using the production 

 2    dual-system estimator, the adjustment that the Bureau was 

 3    considering.

 4         Q.    Dr. Freedman, could you turn to page 2-13 of the 

 5    second's decision.  In the first paragraph under the heading 

 6    "The Quality of the Census Enumeration," the Secretary 

 7    states, "I concede the census's imperfections.  But the 

 8    critical inquiry under this guideline," and I believe is is 

 9    guideline 1, "is not how flawed the census is but whether 

10    the PES can fix it."  

11               Do you agree with that statement of the 

12    Secretary's?

13         A.    Yes, I do.

14         Q.    Why?

15         A.    The question about adjustment is really whether 

16    the PES can fix the problems in the census.  That is a hard 

17    question to answer, but I think that is the question.  For 

18    me the particular focus -- and it is very tough -- is can we 

19    try to get a handle on the errors in the census, can we get 

20    a handle on the extent to which adjustment will fix those 

21    errors without introducing errors of its own.  

22               It is, I think, of not much help to be pointing 

23    to errors, especially gross errors, in the census.  I don't 

24    think that, at least to my mind, advances the inquiry as to 

25    whether or not the PES is good enough to fix the problem in 
                                                              2365

 1    the census.

 2         Q.    Dr. Freedman, can you briefly describe how the 

 3    post-enumeration survey works?

 4         A.    I will take a shot at it.

 5         A.    The post-enumeration survey was a sample of 

 6    blocks.  A block is the minimal unit of census geography.  

 7    There are about 7 million blocks in the country.  The post- 

 8    enumeration survey was a sample of about 12,000 of those.  

 9               You go out into the field with the post- 

10    enumeration survey and try to get at the gross omissions, 

11    that is, people who should have been counted in the census 

12    but weren't, and the erroneous enumerations, that is, people 

13    who were counted in the census but shouldn't have been.  It 

14    is more complicated than that, but in essence the undercount 

15    is the difference between the gross omissions and if 

16    erroneous enumerations.  

17               What you get from the post-enumeration survey 

18    after the field work and the matching, and so on and so 

19    forth, are completed is a set of raw adjustment factors.  

20    That is an estimate of -- let me try and say that 

21    differently.  You are trying to estimate the true 

22    population.  The raw adjustment factor is your estimate of 

23    the multiplier you need to apply to the census population in 

24    order to get the true population, the counts of people 

25    living in the United States as of April 1, 1990, census day.  
                                                              2366

 1               The post-enumeration survey does this for 1392 

 2    poststrata, which are smaller and to some extent more 

 3    homogenius subgroups of the population.  

 4               Maybe that is enough of an answer.

 5         Q.    In principle, couldn't an adjustment be done 

 6    using the raw adjustment factors?

 7         A.    Yes, it could.

 8         Q.    Is that the adjustment that the Census Bureau 

 9    recommended to the Secretary?

10         A.    No, it is not.

11         Q.    What is wrong with using the raw adjustment 

12    factors?

13         A.    The raw adjustment factors suffer from sampling 

14    error.  There is quite a lot of sample error in those 

15    factors.

16         Q.    What is sampling error?

17         A.    The simplest way to say it, I think, is the 

18    sampling error is the difference between the part and the 

19    whole.  If you take 5,000 or 12,000 blocks from the whole 

20    country, that cannot perfectly represent all 7 million 

21    blocks.  There is some difference between the part and the 

22    whole.  

23               So the difference between the thing that you are 

24    trying to estimate and the estimate based on the sample, the 

25    part of that that is driven by the luck of the draw in the 
                                                              2367

 1    sampling, is what we call sampling error.  There are other 

 2    components of error too, but I am just trying to talk about 

 3    sampling error here.

 4         Q.    How is sampling error measured?

 5         A.    There are two conventional ways to measure it in 

 6    statistics.  They are very closely related to each other.  

 7    The standard error is a direct measure of sampling error.  

 8    Then there is another quantity called the variance, which is 

 9    the square of the standard error.  There are other things, 

10    too, but those I think are the two most relevant.

11         Q.    Could you turn, please, to Defendants' Exhibit 

12    64.  I would like you to actually open it up to page 2.  Do 

13    you see Table 1?

14         A.    Yes.

15         Q.    Does Table 1 tell you anything about sampling 

16    error in the PES?

17         A.    Yes, it does.  Take, for example, New York City.  

18    There are something like 33,000 census blocks in New York 

19    City.  The post-enumeration survey sampled 133 of them.  133 

20    blocks cannot be a perfect microcosm of 33,000 blocks, or at 

21    least that would be quite unlikely.  So that illustrates the 

22    problem of sampling error.  

23               In New York State the post-enumeration survey 

24    sampled 520 blocks out of 275,000.  Closer to home, or my 

25    home, in Los Angeles the post-enumeration survey sampled 90 
                                                              2368

 1    blocks out of 25,000.  90 blocks are not going to perfectly 

 2    imitate the 25,000.  

 3               Engelwood, which is a city in Los Angeles County, 

 4    is kind of a striking example.  The post-enumeration 

 5    surveyed sampled one block out of 696.  If you tried to make 

 6    an estimate based on one block out of 696, the sampling 

 7    error would just be horrendous.  

 8               This may get us a little bit away from the topic, 

 9    but of course the post-enumeration survey does not estimate 

10    the undercount rate for Engelwood based only on that one 

11    block.  In fact, most of the adjustment for Engelwood is 

12    computed from other parts of Los Angeles County, other parts 

13    of California, maybe even other parts of the Pacific 

14    division.  

15               (Continued on next page) 

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2369

 1         Q.    Dr. Freedman, can you turn to page 7 of 

 2    Defendant's Exhibit 64.

 3         A.    Yes, sir.

 4         Q.    Do you see table 3?

 5         A.    Yes.

 6         Q.    What does this table show?
                                           

 7         A.    This table shows results from the --  

 8               MR. SOLOMON:  Your Honor, I apologize.  I didn't 

 9    mean to interrupt.  

10               THE COURT:  That's okay.  What you have to say is 

11    far more important than anything he has to say.

12               (Laugher) 

13               MR. SOLOMON:  The doctor is looking at this 

14    report.  I didn't have a problem with respect to it first 

15    because I assumed he was looking at it for purposes of 

16    refreshing his recollection as to what the numbers were.

17               He is here now purporting to tell us what this 

18    table shows which requires him telling us what his analysis 

19    was, and there is no basis for this testimony on the 

20    foundation that has been laid.

21               In addition --  that's all, I will stop.  

22               THE COURT:  What table are we looking at?  

23               MR. SITCOV:  We are looking at page 7, your 

24    Honor, table 3.

25               Dr. Freedman has already testified that he did 
                                                              2370

 1    the computations in this document himself.  

 2               THE COURT:  This is his table?  

 3               MR. SITCOV:  Yes, it is.  

 4               THE COURT:  I don't see any unfairness in that.

 5               I will overrule the objection.  

 6    BY MR. SITCOV:

 7         Q.    What does this table show, Dr. Freedman?

 8         A.    This shows results from the PES for New York and 

 9    is computed from Bureau data by me which we provided to them 

10    on those disketts.

11               There are 72 post-strata in New York, 24 of them 

12    are black, 14 of the black post-strata were undercounted, 

13    ten were overcounted.

14               There are 12 Hispanic post-strata in New York, 

15    nine of them were undercounted, three of them were 

16    overcounted.

17               There are 36 post-strata, that is post-strata 

18    from other demographic groups, 18 are undercounted, 18 of 

19    them are overcounted.  

20               There is no consistent pattern of undercount or 

21    overcount in these data, and that points to sampling error 

22    problems in the raw adjustment factors and indicates why the 

23    adjustment factors do not give you such a stable estimate 

24    for the undercount in New York City.

25         Q.    What did the Bureau do with the raw adjustment 
                                                              2371

 1    factors?

 2         A.    The Bureau took the raw adjustment factors and 

 3    certain other ingredients and ran them through the smoothing 

 4    model.

 5         Q.    Could you turn to page 4 of Defendant's 

 6    Exhibit 64.

 7         A.    Yes.

 8         Q.    Do you see table 2?

 9         A.    Yes.

10         Q.    Is that table based on the smooth or the raw 

11    adjustment factors?

12         A.    It's based on the smooth.

13         Q.    The first column in that table is headed 

14    "Estimated change in shares."

15         A.    Right.

16         Q.    What does that mean?

17         A.    Well, let's take Alabama.

18               Alabama had some share of the total U.S. 

19    population based on the census.  If we were to adjust using 

20    the production dual system estimator, the Bureau's method, 

21    Alabama's share in the population would go up a little.

22               The changes in shares are rather small and so I 

23    am electing to work in parts per million.  I hope that 

24    doesn't sound too much like a chemistry experiment.

25               The change in share for Alabama would be 79 parts 
                                                              2372

 1    per million, so Alabama goes up a little.

 2               Alaska would go down a little, four parts per 

 3    million.

 4               And the table reports those data all the way down 

 5    to Wyoming, which would go up to 11 parts per million.

 6         Q.    The next column says "Confidence intervals." 

 7               What is a confidence interval?  

 8               MR. SOLOMON:  Your Honor, I don't want to keep 

 9    doing this.  This is the point that I have been making.

10               This table, as the note on the next page makes 

11    clear, note 5, does not contain data that if we were to take 

12    the two weeks or two months that the witness said it would 

13    take to get the data out of the disk he gave us we would 

14    find, he says, "The Bureau's published standard errors are 

15    wrong and I have recomputed them using the Bureau's 

16    smoothing model; I have recomputed the shares, too." 

17               We don't know how he did any of that and we have 

18    not been given that data.  

19    BY MR. SITCOV:

20         Q.    Dr. Freedman, can you explain that?

21         A.    Yes.

22               They have been given the data.  I gave it to them 

23    myself.  It's on that diskett.  And they don't need to get 

24    it from me either, all that was available to the public, 

25    there was a notice in the Federal Register and anybody could 
                                                              2373

 1    have responded to that and got the data.

 2               All you do is take the smooth adjustment factors 

 3    that the Bureau provided and you adjust the population and 

 4    you compute the standard errors using the covariance 

 5    matrices.  That's not three weeks work, that is maybe three 

 6    day's work, because you don't get the program right the 

 7    first time.  

 8               MR. SOLOMON:  It is only sitting here now that we 

 9    learned that that is how he has done that.

10               I think it's completely unfair.  

11               THE COURT:  Well, no harm has been done, though, 

12    you now know how it was done, so we will continue.  

13    BY MR. SITCOV:

14         Q.    I believe, Dr. Freedman, you were going to 

15    explain what a confidence interval is.

16         A.    It's harder than explaining how to do the 

17    computations.

18               A confidence interval is a range and the hope is 

19    that the true thing that you are trying to estimate will 

20    fall inside that range.

21               So let's keep on with Alabama, the first line in 

22    the table, that is 79 parts per million share change is an 

23    estimate.  

24               We have a census, we have the adjustment based on 

25    the PES.  There is somewhere unknown to us a true share for 
                                                              2374

 1    Alabama and we want an interval which will cover, include 

 2    that true share, and I have used a standard 95 percent 

 3    confidence interval which, for Alabama, runs from minus 32 

 4    to 189.

 5               The way you compute that is you take the estimate 

 6    and you add and subtract two times the standard error, and 

 7    when you do that, when you follow that procedure, and I'm 

 8    going to assume that the dual system estimator is unbiased 

 9    and I'm going to assume that the Bureau's standard errors 

10    are right --  those are two big assumptions --  you have a 

11    95 percent chance of getting an interval which will cover 

12    the truth.

13         Q.    In the fourth column, there is a heading "T 

14    statistic." 

15               What is a T statistic?

16         A.    Well, it's a question whether the change in 

17    share, the estimated change in share for Alabama reflects a 

18    real differential undercount in Alabama or it just reflects 

19    the sampling error in the smoothed adjustment factors that 

20    we were talking about.

21               The T statistic tries to get at that question in 

22    a way that is quite standard in statistics.

23               You take the estimated change in share, in this 

24    case 79, and you divide by 55 and you get the T statistic of 

25    1.42.  
                                                              2375

 1               And if I may just volunteer something here, if 

 2    you check with a pocket calculator you probably won't get 

 3    exactly 1.42 because these numbers were all computed inside 

 4    the computer with more decimal places than I am showing and 

 5    I rounded it off independently.

 6               Anyway, the T statistic is the estimate divided 

 7    by the standard error.

 8               For Alaska, the T statistic is minus.41.

 9               As T statistics go, those are kind of small for 

10    California, which is represented here today, I believe, and 

11    which is my home state, the T statistic is larger, 4.16, and 

12    the T statistics are reported for all 50 states and D.C.

13         Q.    Is there a convention in statistics about a 

14    cutoff for T statistics?

15         A.    Yes.  For this kind of question, I think it is 

16    quite conventional to say if the T statistic is below 2 in 

17    absolute value, that is, it is between minus 2 and 2, there 

18    is no real evidence of anything going on except sampling 

19    error.

20               If T is bigger than 2 or T is less than minus 2, 

21    you have some evidence --  I mean, in my field I think we 

22    would call that reasonable evidence, not terribly strong, 

23    but reasonable evidence --  that something is going on 

24    beyond sampling error.

25               Now, I don't want to be advocating a hard and 
                                                              2376

 1    fast cut and dried line here, but that's certainly the 

 2    standard convention in the field.

 3         Q.    When you say nothing going on beyond sampling 

 4    error, what do you mean?

 5         A.    Well, let's try New Jersey.  They got an 

 6    estimated share change of 228 parts per million downwards.  

 7    They would lose population share as a result of an 

 8    adjustment.

 9               The T statistic for them is minus 1.62.  That's 

10    within the range that usually would be considered as saying 

11    that the estimate reflects sampling error.  

12               We don't have any real evidence in the data that 

13    New Jersey's share of the population should be changed.  

14    There is no real evidence in the data of a differential 

15    overcount.  It could be there, but the data aren't really 

16    telling us that in any very strong way, and my table here 

17    sort of reports that in two different ways:

18               One is the T statistic of minus 1.62, which is in 

19    that range of minus 2 to 2, the other thing is the 
                                                         

20    confidence interval goes from minus 509 to plus 64 and that 

21    includes zero, so there is nothing in the data which would 

22    tell me I should really believe that New Jersey's share in 

23    the census was wrong.

24               I mean, actually, if New Jersey was suing to 
      prevent an adjustment, I would be kind of happy to testify 
25    for them
                                                              2377

 1         Q.    Does table 2 show anything about the 

 2    effectiveness of smoothing?

 3         A.    I'm sorry, I didn't hear the question.

 4         Q.    Does table 2 show the effectiveness of smoothing?

 5         A.    Yes, I believe it does.

 6         Q.    What is that?

 7         A.    I think table 2 shows that even after you smooth, 

 8    there is still a lot of sampling error left in the estimated 

 9    adjustments for the states.  So we still have a adjustment 

10    with a lot of sampling error in it, not for each and every 

11    state, but in many of those states.

12         Q.    Could you turn to page 6 of Defendant's 

13    Exhibit 64.

14         A.    Yes.

15         Q.    You see there figure 1?
                                        

16         A.    Yes.

17         Q.    What does figure 1 show?

18         A.    Figure 1 shows the same data as in table 2.  The 

19    vertical line represents zero, no adjustment, the horizontal 

20    bars represent confidence intervals that are given 

21    numerically in the table.  They are sorted so that the big 

22    adjustments are on top, the negative ones are on the bottom, 

23    and you can see that California really has quite a large 

24    adjustment, relatively speaking, and that confidence 

25    intervals misses zero.
                                                              2378

 1               There are several states in the top where the 

 2    confidence interval misses zero.  And on the bottom is 

 3    Pennsylvania, a downward adjustment, the confidence interval 

 4    misses zero.  

 5               It is kind to explain the results in Pennsylvania 

 6    on the basis of sampling error alone.  However, for many, 

 7    for a majority of the states, the confidence interval 

 8    crosses zero.  For those states there is nothing in the data 

 9    that really pushes you to thing that those adjustments 

10    reflect real differential undercounts or overcounts.  A lot 

11    of those adjustments could just be reflecting sampling error 

12    in the PES.

13               And I have to say that I am taking what I think 

14    is a very conservative approach to computing the sampling --  

15    the standard errors here.  I'm using the Bureau's theories 

16    in the matter and I am ignoring in these calculations all 

17    kinds of other sources of error in the adjustment.  Those 

18    are just not represented in this figure.

19         Q.    What opinion, if any, do you draw from figure 1 

20    here on page 6 and table 2 on page 4?

21         A.    I think sampling error in the adjustment, even 

22    after smoothing, is a major obstacle to adjustment.

23         Q.    In your opinion, have the results in those two 

24    tables enough to decide not to adjust?  

25               MR. SOLOMON:  Which two tables?  
                                                              2379

 1               MR. SITCOV:  Figure 1 and table 2.

 2         A.    I don't know that I would be prepared to give a 

 3    definitive opinion on that.  If I had time to do more work 

 4    there are other things I would like to think about.  But I 

 5    certainly thing these --  this table and this figure make 

 6    quite a strong argument against adjustment.

 7               If that's all we had, I would be inclined not to 

 8    adjust on the basis of that table and that figure.

 9         Q.    Now, a moment ago you used new Jersey as an 

10    example to illustrate what table 2 is showing.

11               Were you in court during the testimony of 

12    Franklin Fisher?

13         A.    Yes.

14         Q.    Did you hear Dr. Fisher express any opinions 

15    about making tests for each state separately?

16         A.    Yes.

17         Q.    What did you hear Dr. Fisher say?

18         A.    He thought that was a very bad misuse of 

19    hypothesis testing.

20         Q.    Do you agree with Dr. Fisher?

21         A.    No, I don't.  I think it's kind of a reasonable 

22    use of hypothesis testing and that's why I have this table 

23    and this figure in my report.

24               He seems to be arguing, if I heard him correctly, 

25    or he seems to be interpreting these arguments as saying 
                                                              2380

 1    that there is absolutely nothing going on for any state, and 

 2    I don't think that --  that's not what I am trying to do 

 3    here at any rate.  

 4               My argument is only that for many states, for a 

 5    majority of the states, the estimated adjustments are not 

 6    very solid on the grounds of sampling error alone, never 

 7    mind all the other problems.

 8         Q.    Dr. Freedman, could you turn to page 2-32 of the 

 9    Secretary's decision.  That's, again, Plaintiff's Exhibit 9.  

10               THE COURT:  What page?  

11               MR. SITCOV:  2-34.  I'm sorry.

12         Q.    Do you see there, Dr. Freedman, the paragraph 

13    that begins, "Loss functions"?

14         A.    Yes.

15         Q.    Do you see there the Secretary says, "To get a 

16    glimpse of this complexity, it is useful to look at the 

17    undercount rates state by state," and then he has a little 

18    discussion of some of the different results on a state by 

19    state basis.

20               Do you consider that to be a reasonable thing for 

21    the Secretary to have done?

22         A.    Yes.

23         Q.    Are you familiar with the process of presmoothing 

24    as it was used in the 1990 PES?

25         A.    Yes.
                                                              2381

 1         Q.    What is presmoothing?

 2         A.    Well, that's not a question with a short answer, 

 3    but I'll try to keep it down.

 4               It is a process for changing, revising the 

 5    variances, vary the standard errors that the Bureau has for 

 6    the raw adjustment factors.

 7               This is something that is done separately for 

 8    each region in the country.  There are four regions and then 

 9    the American Indian and post-strata are treated separately.

10               When you start out with the post-enumeration 

11    survey, there is a standard statistical technique called the 

12    jackknife --  Dr. Rolph mentioned that, I believe --  for 

13    estimating the variances and even the covariances for the 

14    raw adjustment factors.

15               The variances measure the size of the sampling 

16    error on their own scale, the covariances describe the 

17    statistical interrelationships between the sampling errors, 

18    and the jackknife is quite widely applied in situations like 

19    this.

20               What the Bureau did, and after having that 

21    estimated --  I'm going to call it the raw variance 

22    covariance matrix --  it took a look at the variances.  It 

23    set aside the biggest variances.

24               Then it developed a regression model for the 

25    remaining variances.  That regression model differs from 
                                                              2382

 1    region to region.

 2               Then using that regression model, you predict the 

 3    variances.  You discard the real variances and replace them 

 4    by their predicted values.

 5               Then you multiply the predicted values by some 

 6    number so that the sum of the predicted value is the sum of 

 7    the real value and then you throw back in those big 

 8    variances which you previously set aside.

 9               We have now finished the process of presmoothing 

10    the variances.

11               Then you need these things that are called 

12    covariances and you recompute, if you are the Bureau, you 

13    recompute the covariances to preserve something called the 

14    correlation coefficient, which is a measure of 

15    interrelatedness of statistical things.

16               That's probably a long nutshell, but in a 

17    nutshell, that's what presmoothing is.

18         Q.    Why was presmoothing used?

19         A.    I think presmoothing was used because if you 

20    smooth the adjustment factors without presmoothing, the 

21    undercount goes away, substantially reduced.

22               There is some argument about whether presmoothing 

23    does or does not reduce sampling error.

24         Q.    In your opinion, is smoothing a standard 

25    statistical technique?
                                                              2383

 1         A.    No, I do not believe it is.

 2               I am told that they did it in St. Louis in 1988.  

 3    I personally never heard of it before the spring just before 

 4    the Secretary made his decision, so I don't think that is a 

 5    standard technique at all.

 6         Q.    Were you in the courtroom when John Rolph 

 7    testified about smoothing and presmoothing?

 8         A.    Yes.

 9         Q.    Dr. Rolph testified that modeling variance is a 

10    widely used technique in statistics.

11               In your opinion, is presmoothing --  is the 

12    presmoothing that was used by the Bureau a widely used 

13    statistical technique?  

14               THE COURT:  Would you go back and repeat that 

15    again?  I'm not sure you read whatever you are reading from 

16    correctly.

17         Q.    Dr. Rolph testified that modeling variances is a 

18    widely used technique in statistics.  

19               THE COURT:  Modeling variances?  

20               MR. SITCOV:  Yes, modeling variances.

21         Q.    Is the presmoothing that was used by the Bureau 

22    that you just described a widely used statistical technique?

23         A.    No, I don't believe it is.

24               I don't want to be continuously disagreeing with 

25    John Rolph in this way.
                                                              2384

 1               Variances do get modeled, but they do not get 

 2    modeled the way the Bureau modeled them.  And, in fact, I 

 3    also heard Dr. Rolph testify that, as best I can remember 

 4    his testimony, he couldn't think of an example where 

 5    somebody had computed variances and covariances by the 

 6    jackknife technique and then run them through a regression 

 7    model.

 8               That is what is special about the presmoothing 

 9    that the Bureau did.  

10               MR. SITCOV:  Your Honor, may I approach the 

11    witness?  

12               THE COURT:  Sure.

13               (Handing to the witness)

14               (Pause)

15         Q.    Dr. Freedman, I'm handing you a copy of 

16    Plaintiff's Exhibit 142.

17               Have you seen that before?

18         A.    Yes, I have.

19         Q.    By the way, Dr. Freedman, presmoothing is a form 

20    of modeling variances, is that right?  

21               THE COURT:  That's what troubled me about your 

22    question.

23         A.    It hadn't occurred to me to think of it like that 

24    until I heard the testimony, but, yes, I think you could 

25    describe it that way.
                                                              2385

 1         Q.    Have you seen this letter before?

 2         A.    Yes.

 3         Q.    Could you turn to page 2 of the letter?

 4         A.    I'm there.

 5         Q.    Do you see that there is a little chart there in 

 6    the middle of the page?

 7         A.    If we could call it a table.

 8         Q.    Table.  Fine.

 9               What do you understand the letter in the table to 

10    show about the effect of presmoothing on the size of the 

11    undercount derived from the dual system estimates?

12         A.    The way I understand the argument here, if you 

13    smooth without presmoothing, that lowers the estimated total 

14    national undercount from 2.1 percent to 1.15 percent, and 

15    that advanced an argument in favor of presmoothing.

16         Q.    Do you think that to be an argument in favor of 

17    adjustment?

18         A.    I do not.

19         Q.    Why not?

20         A.    This seems to me to demonstrate a lack of 

21    robustness in the modeling system, that is, a decision 

22    whether to presmooth or not to presmooth seems to have a 

23    very considerable influence on the results of the models, 

24    and to me that is an argument that you should not be using 

25    these models unless you can validate their assumptions 
                                                              2386

 1    because the results depend strongly on those assumptions.

 2         Q.    In your opinion, is it important to consider the 

 3    size of the standard errors of the smoothed adjustment 

 4    factors?

 5         A.    Yes, it is.

 6         Q.    Why?

 7         A.    The question we are struggling with is whether or 

 8    not we can use these adjustment techniques to fix errors in 

 9    a census.

10               Adjustment will remove perhaps certain errors in 

11    the census at the expense of introducing other kinds of 

12    errors into the adjusted counts.

13               One of the yardsticks for gauging all that is the 

14    standard errors in the smoothed adjustment factors, because 

15    they tell you how much sampling error will be in there in 

16    the adjustment.  So I think the standard errors of the 

17    smoothed adjustment factors are one of the critical 

18    ingredients in this comparison we are trying to make.

19         Q.    How are the standard errors of the smoothed 

20    adjustment factors computed?

21         A.    Well, we begin with that raw variance covariance 

22    matrix that came from the jackknife and the raw adjustment 

23    factors.

24               We adjust those raw variances by means of the 

25    presmoothing so we now have the presmoothed variance 
                                                              2387

 1    covariance matrix.

 2               We take the presmoothed variance covariance 

 3    matrix and the adjustment factors and we feed all that 

 4    through the smoothing model and at the other end we get two 

 5    kinds of things:

 6               We get the smoothed adjustment factors and we get 

 7    what the Bureau calls the final variance covariance matrix 

 8    for the smoothed adjustment factors, that being one of the 

 9    huge files on diskett C that we gave to the plaintiffs the 

10    other day.

11               If you are a statistician, you can look at the 

12    variance covariance matrix and pick off the standard errors 

13    you find what is called the diagonal of the matrix.  

14               Matrix is like a giant square array of numbers, a 

15    giant chess board and you look down at the principal 

16    diagonal of the chest board and you pick off the variance 

17    and take the square roots and those are the standard errors.

18         Q.    Do the standard errors of these smoothed 

19    adjustment factors depend on the assumptions in the 

20    smoothing model?

21         A.    They surely do.

22         Q.    Would you turn to Defendant's Exhibit 62, please.  

23    I am going to direct you to the back of that exhibit.  There 

24    is Appendix B which appears to be about ten or so pages in 

25    from the back.
                                                              2388

 1               I would also like you to turn to Defendant's 

 2    Exhibit 66, and in particular to enclosure 3, which, for 

 3    plaintiffs' ease and the court's ease I can tell you and the 

 4    part we are concerned about is represented by this panel.  

 5               MR. SITCOV:  It may be somewhat easier for your 

 6    Honor.

 7               Can you see this?  

 8               MR. SOLOMON:  No.  

 9               THE COURT:  Might this by the time to take a 

10    break?  

11               MR. SITCOV:  Sure.  

12               THE COURT:  It looks like we are about to set 

13    sail into some deep water.

14               Let's take a ten minute break.

15               (Recess) 

16               THE COURT:  Mr. Sitcov.  

17               MR. SITCOV:  Thank you, your Honor.

18    BY MR. SITCOV:

19         Q.    Dr. Freedman, I believe when we left I asked you 

20    if the standard errors of the smooth adjustment factors 

21    depend on the assumptions of the smoothing model, and I 

22    believe you said they do.

23               I would like you to turn to Appendix B of 

24    Defendant's Exhibit 62.

25               Do you see it?
                                                              2389

 1         A.    Yes, sir.

 2         Q.    And what does Appendix B contain?

 3         A.    Appendix B lays out the two main assumptions in 

 4    the Bureau's smoothing model.  The first one is on page 1 

 5    and the second one is on page 2.

 6         Q.    Can you explain the equation number one that 

 7    appears on page 1 of Appendix B?

 8         A.    Well, I'll make a try.

 9               The first assumption in the model says that for 

10    each post-stratum, the raw adjustment factor as estimated 

11    from the post-enumeration survey is equal to the true 

12    adjustment factor plus a random error, and then there are 

13    some --  sorry.

14               The true adjustment factor is what you would have 

15    to multiply the census count in the post-stratum by in order 

16    to get the true population in that post-stratum.

17               Then there are some assumptions about that random 

18    error term that are laid out towards the bottom of the page.

19               The first assumption is that those random errors 

20    have mean zero, which comes down to saying that the raw dual 

21    system estimator, that's the one before you smooth, is 

22    unbiased.

23               The second assumption is that the covariance 

24    matrix for those random errors is equal to the Bureau's 

25    presmoothed covariance matrix.
                                                              2390

 1               The covariance matrix is the thing that describes 

 2    the size and interrelationship among the random errors.

 3               There is a third assumption which I didn't write 

 4    down, it's more of a technical nature, that the random 

 5    errors follow the normal distribution.  That's that bell 

 6    shaped curve that is piled up in the middle and tapers down 

 7    at the ends.  

 8               MR. SITCOV:  Your Honor, may I enter the well for 

 9    a moment?  

10               THE COURT:  Yes, sure.

11         Q.    Dr. Freedman, I would like you to take a look at 

12    what is page 7 of Defendant's Exhibit 66, and I will direct 

13    your attention to the fourth line from the bottom.

14               Is the equation you described to me represented 

15    here?

16         A.    Yes.

17         Q.    And can you explain the notation?

18         A.    Yes.

19               My eyes are terrible, but I think they are almost 

20    --  I can almost read the chart.

21               If you go four lines from the bottom, you will 

22    see an equation that says capital Y is queal to little y 

23    plus e, and then following that there is a string of symbols 

24    which begins with e and ends with N, O, V sub 3 and in 

25    between there is something which we sometimes called 
                                                              2391

 1    twiddles and called wiggly line.  At Berkeley that wiggly 

 2    line is called twiddles, so if I may I will call it twiddles 

 3    here.  

 4               THE COURT:  Twiddles? 

 5               THE WITNESS:  Yes, sir.  That's my testimony.  

 6               THE COURT:  Didn't you once say it was twaddle? 

 7               THE WITNESS:  Now that you have refreshed my 

 8    recollection.

 9               (Laugher)

10         Q.    What part of this states the assumptions that you 

11    were talking about?

12         A.    Well, the equation capital Y equals little y plus 

13    e is that equation in the middle of page 1 of Appendix B, 

14    raw adjustment factor equals true adjustment factor plus 

15    random error.

16               The next thing, e, I will continue to say 

17    twiddles, if I may, N zero V 3 is the --  corresponds, 

18    perhaps I should say, to assumptions one and two on page 1 

19    and this other assumption of normality.

20               The e is the set of random errors, twiddles more 

21    formally means distributed as, capital N on the chart means 

22    normal, zero on the chart means mean zero, so that's what 

23    assumption one on page 1 is, and then the rest of it, the V 

24    sub 3, says that the variance covariance matrix for the 

25    errors is V sub 3, which happens to be the presmoothed 
                                                              2392

 1    variance covariance matrix.

 2               There is a little conflict of terminology here.  

 3    Some of us call it presmooth, some of us call it smoothed.  

 4    I think on page 7 it's referred to as the smoothed estimated 

 5    sampling covariance matrix up around the middle of the page.

 6         Q.    Could you turn to page 2 of Appendix B.

 7         A.    Yes.

 8         Q.    And what is the material on page 2 of Appendix B?

 9         A.    Well, that's the second major assumption in the 

10    Bureau's smoothing model.  The idea there is that the true 

11    adjustment for a post-stratum can be expressed as a linear 

12    combination of explanatory variables with an add-on random 

13    error.

14               Maybe I should start with the linear combination 

15    of explanatory variance part.

16               The Bureau has a procedure for picking 

17    explanatory variables with the mail return rate or the 

18    percentage of people in a post --  sorry --  so that would 

19    be the percentage --  let me use the other one --  renters 

20    --  the percentage of people in a post-stratum who rent or 

21    the percentage of people in a post-stratum who live in 

22    multiunit housing.  Those would be typical explanatory 

23    variables.

24               Then there are coefficients, and the way you get 

25    that linear combination is you go through the sequence of 
                                                              2393

 1    coefficients and explanatory variables, one coefficient for 

 2    each covariance, and you add them up and that gets you the 

 3    central term, linear combination of explanatory variables.

 4               Then there is some random force, this random 

 5    error term, which drives the true adjustment factor off the 

 6    linear combination of explanatory variables.

 7               Now, the assumptions about the random error term, 

 8    one, the errors are supposed to have mean zero; two, the 

 9    errors are supposed to be independent of each other and of 

10    the errors in the first equation that was described on the 

11    previous page.

12               Assumption three her notes that the explanatory 

13    variables are different in the different regions of the 

14    country, the four regions.  The coefficients, however, are 

15    assumed to be constant within each region.

16               The last assumption on the page, assumption five, 

17    is that the average size of those random errors is constant 

18    across post-strata within a region, and technically the 

19    average size that I am talking about is root mean square or 

20    standard deviation.

21               There is another yet more technical assumption 

22    that these errors follow the normal distribution, and I did 

23    not write that down on the page 2.

24         Q.    I would like to direct your attention back to 

25    this blow-up of page 7 of Defendant's Exhibit 66.
                                                              2394

 1               Do you see the third line from the bottom?

 2         A.    Barely.

 3         Q.    Does that state algebraically the assumption or 

 4    the equation and assumptions you just described from page 2 

 5    of Appendix B?

 6         A.    Yes.

 7               The first thing on the left on that line says 

 8    that little y is equal to capital X times Beta plus V.

 9               Little y is the vehicle for true adjustment 

10    factors, capital X represents the explanatory variables, 

11    technically it's a matrix.

12               Beta represents the coefficients, those things 

13    that were assumed to be constant within a region.

14               X times Beta is an algebraic way of saying that 

15    the linear combination of explanatory variables.

16               V represents the random error terms.

17               The last part of the equation --  it's not really 

18    the last part of the equation, the last part of that line 

19    says that V is normally distributed with mean zero and 

20    variance covariance matrix, sigma squared times the 

21    identity.  That says that the errors had mean zero, they are 

22    independent, they had constant size sigma and it follows 

23    normal distribution.

24         Q.    Dr. Freedman, you testified earlier that you --

25         A.    Sorry, if I may.
                                                              2395

 1               Then at the bottom of the chart it says that e 

 2    and V are independent, and that corresponds to my assumption 

 3    2 here.

 4               And then there is a carryover sentence, V 3 is 

 5    estimated, but is, and I think on the next page it's 

 6    something like it's considered as known, and that belongs 

 7    with page 1 of Appendix B.

 8         Q.    I believe earlier you testified that you appeared 

 9    as a witness in the Cuomo Baldridge litigation, right?

10         A.    Yes.

11         Q.    What was the subject of your testimony in that 

12    case?

13         A.    The main subject of the testimony was smoothing 

14    models.

15               Plaintiffs in that case, who include at least 

16    some of the plaintiffs in this case, had as experts, Jay 

17    Kadane, who is not here, Gene Ericksen and Franklin Fisher, 

18    who appeared here, and the three of them had a smoothing 

19    model which was rather like this model and they wanted to 

20    adjust the 1980 census using that model, and I came in for 

21    the government side to talk about that model.

22         Q.    How, if at all, is the 1990 production smoothing 

23    model like the 1980 smoothing model?

24         A.    Well, I think of them as being very similar, so 

25    it might be easier for me if I could just point to the 
                                                              2396

 1    differences.

 2               In 1980, the modeling was being done on 66 

 3    geographical areas, here its being done on 1,392 post-strata 

 4    grouped into these four regions and the American Indians, so 

 5    that's one difference.

 6               In the first equation in 1984, the assumption was 

 7    being made that the random errors were independent from one 

 8    geographical area to another.

 9               In 1990 its being assumed that those errors have 

10    the covariance matrix that the Bureau got by the 

11    presmoothing, so that's a difference.

12               In 1984, one set of explanatory variables was 

13    being used and in 1990 quite a different set is being used, 

14    and I think those are the main differences that I see.

15         Q.    In your opinion, are the assumptions in the 1990 

16    production smoothing model reasonable?

17         A.    No, I don't believe they are.

18         Q.    Why don't you think they are reasonable?

19         A.    Well, let's start with the assumptions on page 1 

20    of Appendix B of I guess it's Exhibit 62.

21               The first assumption is that the dual system 

22    estimator, the raw dual system estimator, the raw adjustment 

23    factors are unbiased, and the Bureau has done quite a lot of 

24    work on that in the P studies and has tried to measure the 

25    bias in the dual system estimator and it is appreciable.  
                                                              2397

 1               It depends how you define things, but either 25 

 2    percent or 33 percent by their reckoning of the estimated 

 3    undercount is accounted for by bias, and I think they have 

 4    subsequently revised that estimate upwards by a very 

 5    substantial amount.  

 6               MR. SOLOMON:  Your Honor, there is no foundation 

 7    for that, just rank hearsay.  I move to strike the last 

 8    piece of the witness' answer.  

 9               MR. SITCOV:  He certainly can testify to hearsay 

10    if it's the kind of thing that an expert would normally rely 

11    on.  

12               THE COURT:  What was the last statement that 

13    triggered this?  It went right by me.  

14               MR. SOLOMON:  He made reference to what he thinks 

15    might be some revision by the Bureau with respect to 

16    estimates of error.

17               It hasn't been established that he has any basis 

18    at all, and here we are getting into --  

19               THE COURT:  Is this speculation that you are 

20    engaging in? 

21               THE WITNESS:  No, sir, it is not.  

22               THE COURT:  Would you like to tell me why it's 

23    admissible? 

24               THE WITNESS:  Your Honor, I can't tell you why 

25    it's admissible, I can tell you where I heard it.  
                                                              2398

 1               THE COURT:  You heard it somewhere? 

 2               THE WITNESS:  I heard it and I read it.  

 3               THE COURT:  In the Census Bureau document or -- 

 4               THE WITNESS:  I read it in a document and I heard 

 5    it from a Census Bureau official.  

 6               THE COURT:  I will permit it.  I will let you 

 7    bring it out on cross.  I'm not really sure what is going on 

 8    here, but --  

 9               MR. SOLOMON:  Okay.  As long as you promise to 

10    bring it out on cross --  

11               THE COURT:  If I rule otherwise, remind me that I 

12    promised.  

13    BY MR. SITCOV:

14         Q.    I'm sorry, where were you?

15         A.    I was in the witness stand trying to explain why 

16    I thought assumption one in the smoothing model was not 

17    reasonable, and one basis for that opinion was measurements 

18    of bias in the raw adjustment factors.

19               We know those raw adjustment factors have 

20    substantial bias in them, although there is some uncertainty 

21    about how much that bias is.  That was point one.

22               Point two, I cannot believe that the Bureau's 

23    presmoothed covariance matrix actually gives the covariance 

24    of the errors in that equation.

25               In the first place, I rather thing that 
                                                              2399

 1    presmoothing introduces bias.  In the second place, we know 

 2    that presmoothed covariance matrix is subject to sampling 

 3    error.  So that is why I do not find the assumptions in 

 4    equation one so reasonable.

 5               If I may turn to equation two, we know, we think 

 6    we know that census errors vary considerably from 

 7    post-stratum to post-stratum.  After all, that is why people 

 8    are interested in an adjustment to begin with.  And I find 

 9    it remarkably unlikely that the random errors in equation 

10    two could be of the same size.  I find it remarkably 

11    unlikely that they should be unrelated from post-stratum to 

12    post-stratum.

13               Let me try to give a concrete example.

14               If you happen to miss a household in the census, 

15    classical household, you are going to miss a husband, a wife 

16    and 2.2 children, and that will create some kind of 

17    correlation in the error structure which is just not 

18    reflected in these assumptions, because, after all, the 

19    husband, the wife and the children belong to different 

20    post-strata.

21               So to me this equation seems unreasonable.

22         Q.    Do the assumptions have to be exactly true in 

23    order for the model to be useful?

24         A.    No, of course not.  If they are close to true, 

25    close enough to true, then the model would be useful.
                                                              2400

 1         Q.    Have you seen any evidence to demonstrate that 

 2    the assumptions in the 1990 production smoothing model were 

 3    close enough to true for the model to be useful?  

 4               MR. SOLOMON:  Your Honor, if all we are doing 

 5    this afternoon is having this witness comment on the 

 6    evidence, I do object.  If what he is talking about is 

 7    something he may have looked at in order to form his 

 8    opinion, I don't object.  

 9               MR. SITCOV:  That is precisely what we are 

10    getting at, what he has looked at that would form his 

11    opinion.  

12               THE COURT:  The second of the two things?  

13               MR. SITCOV:  Yes.  

14               THE COURT:  All right.  You represent that and I 

15    take it the objection is withdrawn.

16         A.    I have not.

17         Q.    Did any of the Bureau's P studies consider the 

18    extent to which assumption two of the production smoothing 

19    model holds?

20         A.    I don't believe they do.

21         Q.    Are you familiar with the report that the 

22    undercount steering committee submitted to the director of 

23    the Census Bureau?

24         A.    Yes.

25         Q.    Did the undercount steering committee consider in 
                                                              2401

 1    that report the extent to which assumption two of the 

 2    smoothing model holds?

 3         A.    Yes, to some degree.  What they said, as I 

 4    recall, is that this assumption was open to some question 

 5    and they wished they had time to do a sensitivity analysis 

 6    on it, in other words, to change the assumption and see what 

 7    impact that would have on the results.

 8               That is all I remember about the extent of the 

 9    consideration.

10         Q.    Did they actually do that?

11         A.    I don't believe they did.

12         Q.    Have you read the reports that were submitted to 

13    the Secretary by Eugene Ericksen, Leo Estrada, John Tukey 

14    and Kirk Wolter?

15         A.    I have.

16         Q.    And have you read the joint report?

17         A.    I have.

18         Q.    Did any of those reports consider the extent to 

19    which assumption two of the 1990 production smoothing model 

20    holds?  

21               MR. SOLOMON:  Objection.  

22               THE COURT:  Overruled.

23         A.    I don't believe they did.

24         Q.    Did any of the plaintiffs testify about the 

25    extent to which the --  
                                                              2402

 1               THE COURT:  Did any of the plaintiffs?  

 2         Q.    Did any of the plaintiffs' experts testify about 

 3    the extent to which assumption two of the 1990 production 

 4    smoothing model holds?  

 5               MR. SOLOMON:  Same objection, your Honor.  

 6               THE COURT:  Overruled.

 7         A.    I don't believe they did.

 8         Q.    Dr. Freedman, I would like you to look at the 

 9    chart that is up here that is closest to me.

10               Do you see that?

11         A.    Yes.

12         Q.    Do you recognize that document?

13         A.    I do.  I think it would be fair to say I made it 

14    up.

15         Q.    What is it?

16         A.    It is a variation on the page to the right.

17               That is, I took the Bureau's model and I clipped 

18    it apart, I mean, I made several Xerox copies, I clipped it 

19    apart and scotch taped it together to make an alternative 

20    model, and the difference, if you go up three lines from the 

21    bottom on each chart, there is a math display that ends, and 

22    start on the right, which is the Bureau's page, you see the 

23    letter capital I, and in the chart on the left you see 

24    capital V sub 3, and I hate to tell you, but a lot rides on 

25    whether you put a capital I there or a capital V sub 3.
                                                              2403

 1         Q.    What is the nature of the alternative assumption 

 2    that you have made?

 3         A.    The Bureau's model assumes that errors in the 

 4    second equation are independent from one post-stratum to 

 5    another and they have the same average size.

 6               The alternative assumption that I created in that 

 7    way assumes that the errors are related from one 

 8    post-stratum to another and have different sizes and there 

 9    is an empirical base, although a rather weak one, for the 

10    assumption about the sizes and the interrelationship, 

11    namely, they are assumed proportional to the Bureau's 

12    presmoothed covariance matrix.  That's the V sub 3.

13         Q.    In your opinion, is the alternative that you just 

14    discussed a reasonable assumption?

15         A.    I would describe it as less unreasonable than the 

16    original assumption.

17         Q.    Why do you think that?

18         A.    Because the alternative assumption has some 

19    qualitative features that seem more in tune with what we 

20    think about the census, namely, that the errors for the 

21    different post-strata are correlated and have different 

22    sizes.

23               When I say errors here, I mean errors in the 

24    second equations.

25         Q.    Why do you say that something we know about the 
                                                              2404

 1    census?

 2         A.    "Know" is too strong, but it is something we 

 3    think on the basis of these studies, like the 

 4    post-enumeration survey, and they do suggest that there are 

 5    different undercount and overcount rates for different 

 6    post-strata.  The sizes of those things we do not know very 

 7    well, but qualitatively, that seems right to me.

 8         Q.    In your opinion, have the plaintiffs presented 

 9    evidence in this case that shows that the Bureau's model is 

10    closer to the truth than the alternative that you have 

11    developed?  

12               MR. SOLOMON:  Objection.  

13               THE COURT:  Sustained.

14         Q.    Could you turn to Defendant's Exhibit 62, please.  

15    I would like you to turn to page 28. 

16               Did you see about halfway down there there is a 

17    table 8?

18         A.    Yes.

19         Q.    Does this table have anything to do with the 

20    alternative that you propose?

21         A.    It does.

22         Q.    And what does it show?

23         A.    Well, it shows that taking out I and putting in V 

24    sub 3 makes a big difference to the standard errors for the 

25    adjustments.
                                                              2405

 1               What the table shows is standard errors computed 

 2    on two different theories:

 3               The first theory is the theory in the chart at 

 4    the right and the second theory is the theory in the chart 

 5    at the left, and the standard errors computed from the 

 6    alternative model are substantially larger than the standard 

 7    errors computed from the Bureau's model.

 8               For New York there is a factor of 3.  As the 

 9    footnote says, these numbers are computed without a step 

10    called benchmarking.

11         Q.    Why was that?  

12               MR. SOLOMON:  Your Honor, even the witness admits 

13    that we have not received even yet the program that ran this 

14    alternative assumption.  It is improper, we believe, at 

15    least certainly unfair, for him to be able to testify to it 

16    when we cannot replicated his work to see whether he has 

17    done it correctly or not.  

18               THE COURT:  I will overrule the objection, 

19    although I understand your point.

20         Q.    Why were these not benchmarked?

21         A.    Well, I wanted to present results for the regions 

22    in New York, that was a relatively easy thing to do.

23               If you benchmark, then that forces all the 

24    adjustments to be equal to the original PES estimates and 

25    you would not see the difference.  So that, in fact, 
                                                              2406

 1    benchmarking would reduce the difference between the results 

 2    of the Bureau model and the alternative model, although by 

 3    the time you got down to a place like New York City or even 

 4    a smaller place, there would be an appreciable difference 

 5    between the two sets of standard errors.

 6         Q.    In your opinion, does the data reflected in table 

 7    8 say anything about the if he can of the assumptions in the 

 8    smoothing model?

 9         A.    I think it does.

10         Q.    What does it say to you about that?

11         A.    It tells me that the assumptions have a big 

12    impact on the standard errors.  To a considerable extent the 

13    standard errors are driven not by the data, but by the 

14    assumptions.  At least with respect to standard errors, the 

15    smoothing model is not robust.

16         Q.    In your opinion, which way does table 8 cut in 

17    the argument about whether to adjust?

18         A.    To me it is an argument against adjusting.

19         Q.    Does that table suggest anything about the 

20    sensitivity of the standard errors to assumptions?

21         A.    Yes.  I think it shows that the standard errors 

22    are quite sensitive to the assumptions in the model.

23         Q.    You mentioned a moment ago that you did not 

24    benchmark these numbers.

25               Did the Bureau's production dual system estimate 
                                                              2407

 1    involve benchmarking?

 2         A.    It did.

 3         Q.    What exactly is benchmarking?

 4         A.    Well, let me back up to the point where we had 

 5    the raw adjustment factors and the presmoothed variance 

 6    covariance matrix and we fed those things through the 

 7    smoothing model, and what you get out is the smoothed 

 8    adjustment factors and something else which I am going to 

 9    forget about for the moment.

10               Now, if you went ahead and adjusted the northeast 

11    region, for example, using the smoothed adjustment factors, 

12    you would get a different answer than if you had used the 

13    raw adjustment factors, and what the Bureau does is it 

14    forces the two adjustments to agree.  It multiplies the 

15    smooth adjustment factors for the northeast all by the same 

16    constant factor and that constant factor is chosen to have 

17    the effect of making the adjustment be identically the same 

18    whether you use the smoothed factors or the raw factors.

19               That's what benchmarking is.

20         Q.    Is benchmarking in that context a standard 

21    operation?

22         A.    No, I don't believe it is.  In other contexts it 

23    is, but not in this one.

24         Q.    Why isn't it standard here?

25         A.    Well, what I think is that if you believed in the 
                                                              2408

 1    smoothing model, you would like the smooth adjustment 

 2    factors that you got out and you would use them and you 

 3    would not benchmark back to what the raw factors are telling 

 4    you.

 5         Q.    Could you turn to page 21 of that same report?

 6         A.    Yes.

 7         Q.    Do you see table 5?

 8         A.    Yes.

 9         Q.    What does that table show?

10         A.    That table shows the impact of various stages in 

11    the Bureau's modeling procedure.  It shows results for each 

12    region, and perhaps we can start with the northeast, and 

13    each line of the table gives you an estimated adjustment and 

14    the standard error for that adjustment.

15               For example, with the raw post-enumeration 

16    survey, the adjustment for the northeast is 56,000 with a 

17    standard error of 258,000.

18               The next line of the table considers what would 

19    happen if you just smoothed the raw factors without 

20    presmoothing the variances.  And if you do that, the 

21    undercount drops very sharply to 270,000 standard errors 

22    also shown.

23               The next step is to presmooth the variances, but 

24    forget about this business of rescaling them so that the sum 

25    of the variance as predicted from the regression model is 
                                                              2409

 1    equal to the sum of the real variances.  Let's just drop 

 2    that rescaling step for a moment.

 3               The population adjustment shoots back up to 

 4    624,000.  It is now substantially above the raw PES.  The 

 5    standard error is now 143,000.

 6               That's the measure of sampling error in the 

 7    624,000.

 8               The next step is to rescale those predicted 

 9    variances.  That does not change the population adjustment 

10    very much, it makes the standard error go down a little, and 

11    then there is the last step of benchmarking, which makes the 

12    629,000 drop back, the number we got was 564,000 with a 

13    standard error of 225,000.

14               So in the northeast, benchmarking caused the 

15    Bureau to drop nearly 100,000 people from the adjustment.

16               In the south, the change due to benchmarking is 

17    even more dramatic.  There are upwards of 400,000 people who 

18    are added to the population adjustment on the basis of 

19    benchmarking.

20         Q.    In your opinion, does benchmarking have an impact 

21    on the PES results?

22         A.    Yes, I think the statement shows that it does.

23         Q.    Do you regard this table as evidence in favor of 

24    adjustment?

25         A.    No.  It seems to me evidence against.
                                                              2410

 1         Q.    Why is that?

 2         A.    Well, there is a major reason and a minor reason.

 3               The major reason is that you can see from the 

 4    table that the adjustments that you get depend very strongly 

 5    on the details of the modeling procedure.  You have to 

 6    decide whether you want to presmooth the variances or not.  

 7    That makes a big difference.  You have to decide whether you 

 8    want to rescale them or not, you have to decide whether you 

 9    want to benchmark or not.

10               All those decisions create uncertainty in the 

11    final estimate, because we do not really know which of these 

12    sets of detailed decisions are the right ones to make, and 

13    the estimated adjustment is quite sensitive to those 

14    details.  So that's a point.

15               Another point, this is the minor one, is that by 

16    benchmarking, the Bureau forced the final smoothed estimate 

17    to be equal to the original raw estimate for each region.  

18    That would, of course, not be the case for subareas, like 

19    cities or states, but it would be true region by region.

20               In that case the standard error for the final 

21    regional adjustment should be equal to the standard error 

22    for the original regional adjustment, but somehow the Bureau 

23    has made, in the northeast, about ten percent of the 

24    standard error disappear.

25         Q.    In your opinion, do the standard errors that 
                                                              2411

 1    result from the benchmarking capture the uncertainty in the 

 2    estimates?

 3         A.    No, I don't believe they do, that is, they 

 4    perhaps capture some of the sampling error, although even 

 5    there there is, I think, substantial evidence that the 

 6    standard errors are too small, but they certainly do not 

 7    reflect the impact of other uncertainties, such as whether 

 8    you should benchmark, whether you should presmooth and so 

 9    forth, and if we come to talk about subareas of the regions, 

10    like the states or the cities, the standard errors do not 

11    capture the impact of failures in the homogeneity assumption 

12    which would be used in passing the adjustment from 

13    post-strata to geographical areas.

14         Q.    Does this table tell you anything about the 

15    robustness of the smoothing model?

16         A.    I believe it does.

17         Q.    What does it tell you about the robustness of the 

18    smoothing model?

19         A.    I think it points to a lack of robustness both in 

20    the estimates and in the estimated standard errors.

21         Q.    Is the selection of variables to be used in the 

22    smoothing model important?

23         A.    I think it is.

24         Q.    Did the Bureau decide in advance which variables 

25    would go into the production smoothing model?
                                                              2412

 1         A.    I can't give a yes or no answer to that.

 2               They decided in advance on a list of variables 

 3    that had to go into the model and they decided in advance on 

 4    a list of candidate variables for the model.

 5               They also decided on a computerized algorithm, a 

 6    method for deciding which of the candidates would get into 

 7    the equations and which candidates would not get in.  But 

 8    the decision as to which actual variables get into the 

 9    equation, that decision was made on the basis of the 

10    post-enumeration survey data.

11         Q.    Did you hear Dr. Rolph testify for the plaintiffs 

12    about whether the Bureau used proper variables in the 

13    smoothing operation?

14         A.    I did.

15         Q.    Did Dr. Rolph testify that the computer algorithm 

16    used to select the variables from the list you were just 

17    describing --  

18               MR. SOLOMON:  Objection.  

19               THE COURT:  Overruled.

20         Q.    Did you hear him testify about the computer 

21    algorithm used to selected variables from the list of 

22    candidate variables you were just describing?

23         A.    I didn't hear him discuss the algorithm.

24         Q.    Is the Bureau's variables selection algorithm 

25    standard?
                                                              2413

 1         A.    That seems too strong to me.  It is often used, 

 2    but I would not describe it as standard.  Some people like 

 3    it, some people don't.

 4         Q.    Have you considered an alternative selection 

 5    method for variables?

 6         A.    I have.

 7         Q.    Could you turn to page 29 of Defendant's 

 8    Exhibit 62.

 9         A.    Yes.

10         Q.    You see table 9?

11         A.    I do.

12         Q.    What does that table show?  

13               MR. SOLOMON:  I will simply note, your Honor, 

14    what I have been saying throughout, many of these numbers 

15    are based on the same data.  

16               THE COURT:  Are based on the --  

17               MR. SOLOMON:  The same data which we do not have 

18    any programs to replicated.

19         A.    Table 9 compares the Bureau's model with a model 

20    that was developed by Dan Costa, Richard Cutter and myself.  

21    We followed the Bureau's modeling strategy and used their 

22    variables, with one exception --  well, with two exceptions, 

23    really, in table 9.

24               We used an alternative, and I think more 

25    standard, variable selection method and we did not presmooth 
                                                              2414

 1    or benchmark, and the Bureau's model estimates a national 

 2    undercount of 5.27 million --  let's just call that 5 

 3    million --  the main model that we developed, which is 

 4    described here as the alternative model, estimated a 

 5    national undercount of just over 3 million.

 6               We also considered what we call her the simplest 

 7    model.  That is a model that completely gets rid of the 

 8    variable selection problem by having only one variable.  I 

 9    think only a statistician would call it is a variable.  It's 

10    the constant term intercept.  And that model estimates an 

11    undercount of 2.8 million.

12               For New York City, the Bureau's model estimates 

13    an undercount of about 230,000, alternative model gets an 

14    undercount of 98,000, and our simplest model gets an 

15    undercount of 85,000.  Our standard errors compare quite 

16    favorably with the Bureau's.

17               And what this table shows is how the estimates of 

18    the undercount depend on the modeling strategy.  But I do 

19    have to caution that in this table the alternative model and 

20    simplest model do not presmooth and do not benchmark.

21         Q.    In your opinion, does this table say anything 

22    about the robustness of the smoothing model?

23         A.    Yes.

24         Q.    What does it tell you about the robustness of the 

25    smoothing model?
                                                              2415

 1         A.    It points to a lack of robustness, because a 

 2    different variable selection procedure will give different 

 3    answers, so the answers, in part, are driven by the rule you 

 4    use to select variables.

 5         Q.    Do the Bureau's standard errors reflect 

 6    uncertainty created by the variable selection process?

 7         A.    No, I don't believe they do.

 8         Q.    Could you turn to Defendant's Exhibit 63.

 9               What does this exhibit show?

10         A.    Exhibit 63?

11         Q.    Yes, alternative models for smoothing adjustment 

12    factors.

13         A.    Well, that's a report which describes how we 

14    developed these alternative models.

15         Q.    Does that reflect or is your opinion that the 

16    choice of the model has an effect on the undercount estimate 

17    based on the work in this report?

18         A.    In part.

19         Q.    Did you hear Dr. Rolph testify that models like 

20    the Bureau's smoothing model were tested by the Bureau in 

21    the TARO test census in 1986 and in the test census in 1988?

22         A.    I did.

23         Q.    Do you agree with his testimony?

24         A.    Well, in part, but certainly not in the whole.

25         Q.    What is the source of your disagreement?
                                                              2416

 1         A.    Well, the Bureau did do a test census in Los 

 2    Angeles and a test post-enumeration survey and it did run 

 3    the smoothing model or a smoothing model on the data and we, 

 4    therefore, know that the smoothing model runs and produces 

 5    output, and so in that sense the model was tested.

 6               But there is another and I think more interesting 

 7    sense of the word "test" which I would like to illustrate by 

 8    referring to a study discussed by Dr. Rolph.

 9               He told us about his smoothing model about fire 

10    alarms where he fitted that model to three years of data.  

11    The model makes some predictions about whether fire alarms 

12    are false alarms or if I can call them that true alarms, and 

13    then he went and he looked at a fourth year of data where he 

14    knew whether the alarms in that fourth year were false 

15    alarms or true alarms and he could see whether his model was 

16    doing a good job at making those kinds of discriminations.

17               So if I understood his testimony properly, he was 

18    able to test his model against external reality in a 

19    situation where he knew what the right answers were.  

20               And I think that was a very nice study and I 

21    would like to complement him on it.  I think that's what 

22    good science is like, you have a theory and you test it 

23    against some data where you know some answers.

24               But I would also like to contrast the fire alarm 

25    approach, if I can name it that, with what the Bureau did in 
                                                              2417

 1    1986, because in 1986 we don't have any baseline external 

 2    data against which we can test the model.  We do not know if 

 3    those model outputs are in any correspondents with external 

 4    reality, and that is why I do not think the tests in 1986 or 

 5    1988 have much force.

 6    

 7    

 8    

 9               (Continued on the next page) 

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2418

 1         Q.    Dr. Freedman, I would like to turn your attention 

 2    now to the subject of loss functions.  What is it that the 

 3    Census Bureau's loss function analyses tried to do.

 4         A.    I think to put the best face on it, what the loss 
                                                                   

 5    function analysis tries to do is to compare the distributive 

 6    accuracy, that is, the accuracy of population shares as 

 7    determined by the census and as determined by adjustment.

 8         Q.    How does it do that?

 9         A.    That is another question to which I don't have a 

10    short answer.  Let me start in the simplest possible 

11    situation where we actually knew the true population shares.  

12    Of course, we don't.  If we did, we would not be here in 

13    Court today.  But just to get started, let's consider that 

14    situation where you now the true population shares for each 

15    state.  

16               Then you could take, for example, Alabama and 

17    compute its true population share and compute its population 

18    share as determined from the census and look to see how far 

19    off the census was.  You could do that for all 50 states and 

20    D.C. and add up all those errors.  That would get you the 

21    absolute error loss function for the census.  

22               Then you could do a similar thing for adjustment, 

23    and you could compare and see whether the census had a 

24    higher loss than the total error or adjustment did.  

25               Instead of taking the size of the error, 
                                                              2419

 1    statisticians like to square things.  It is more convenient 

 2    in some ways.  You could square the errors and then add them 

 3    up and make the comparison, and you would have -- 

 4               THE COURT:  Some day you can explain to me why 

 5    that is true.  I have been debating that in chambers for two 

 6    weeks now.  Why do you take a number or a bunch of numbers 

 7    that you know you have and fool around with them and square 

 8    them?  And then you back up later on usually by moving 

 9    decimals around?  I never figured that out either.  

10               Anyway, forgive the digression.  

11               THE WITNESS:  May I defend my profession for 30 

12    seconds?  

13               THE COURT:  Certainly.  

14               THE WITNESS:  You are probably going to like this 

15    as much as you like squaring.  The thing is, we do a lot of 

16    work with differential calculus.  It is easier to 

17    differentiate a square than it is to differentiate the 

18    absolute value function.  The square is more convenient -- 

19               THE COURT:  Because the number is larger?  

20               THE WITNESS:  No.  Because it fits in better with 

21    the differential calculus that has to be done in order to 

22    prove the kind of theorems that we like to prove.  So it is 

23    an argument from convenience for us.  

24               THE COURT:  You have left the darkness 

25    unobscured.  
                                                              2420

 1               THE WITNESS:  Let me get back to loss functions.

 2         A.    We are in a situation where we do not have this 

 3    external truth and cannot compute the errors whether we take 

 4    the absolute value or the square.  There are two statistical 

 5    moves to make to get around that difficulty.  

 6               The first move is to change from loss to expected 

 7    loss.  Technically, expected loss is called risk.  

 8               What does "expected" mean?  "Expected" means you 

 9    take the average over all possible realizations of the post- 

10    enumeration survey.  The post-enumeration survey was a 

11    sample of 12,000 blocks out of 7 million.  It was one 

12    possible sample.  There are many possible samples.  When we 

13    say "expected value," we mean on the average over all 

14    possible samples, all the ones that could have been but 

15    weren't K. that is one thing.  

16               The other thing is that truth has to be 

17    estimated.  So you are going to move from loss to expected 

18    loss to estimated expected loss, or estimated risk, as we 

19    call it.  Truth is estimated in the Bureau's loss function 

20    analysis by taking the production dual-system estimator and 

21    correcting it for various kinds of bias.  

22               I think a final point that has to be mentioned is 

23    that in thinking about how far off adjustment is likely to 

24    be from truth, you have to bring in the standard errors or, 

25    more particularly, the variances of the smoothed adjustment 
                                                              2421

 1    factors, or, even more technically, that variance/covariance 

 2    matrix for the smoothed adjustment factors.

 3         Q.    Does the Bureau's loss function measure the costs 

 4    of misallocation of seats in Congress due to mistakes in 

 5    enumeration?

 6         A.    I do not believe that it does.

 7         Q.    Does it measure the cost of the misallocation of 

 8    federal funds due to mistakes in enumeration?

 9         A.    I do not think it does.

10         Q.    So what is the loss it is measuring?

11         A.    I do not think that the loss function is 

12    measuring loss in that sense.  Statisticians like to 

13    appropriate English words and then use them in a technical 

14    sense:  Loss, normal, and so forth.  

15               The only loss that is being considered here is 

16    the error in the shares, whether absolute value or squared 

17    or whatever it may be.

18         Q.    Were you in court when Dr. Franklin Fisher 

19    testified about loss functions?

20         A.    Yes.

21         Q.    Did you hear him testify that the Bureau's loss 

22    functions really measure the social costs of errors in the 

23    census or in the adjustment?

24         A.    No.  I thought he testified in a way much more 

25    similar to mine on that point.
                                                              2422

 1         Q.    Do you know if the Census Bureau planned to use 

 2    loss function analysis as a decision rule?

 3         A.    I believe they did not.

 4         Q.    Could you turn to Plaintiffs' Exhibit 654.  I 

 5    think it is in that large binder.  Do you see the page with 

 6    the Bates stamp number 7857?

 7         A.    I do.

 8         Q.    Is there something on that page that suggests to 

 9    you that the Bureau did not plan to use loss function 

10    analysis as a decision rule?  

11               MR. SOLOMON:  Objection.  

12               THE COURT:  I will permit it.  It is late in the 

13    day.  The 20 of five rule.  

14               MR. SITCOV:  I lost my head. 

15         A.    Yes, I do.  Paragraph 4 says that loss functions 

16    should be thought of as organizational tools, but not the 

17    only set of tools, to assist the Bureau in its review and 

18    adjustment decision-making process.

19         Q.    In your opinion, should the adjustment decision 

20    have been based solely on loss function analysis?

21         A.    No.  I think I would have objected quite strongly 

22    to that.

23         Q.    Does the loss function analysis that the Census 

24    Bureau conducted favor adjustment?

25         A.    On its face, yes.
                                                              2423

 1         Q.    Does the Bureau's loss function analysis tell you 

 2    how much improvement in accuracy can be expected from 

 3    adjustment?

 4         A.    Yes.  That is what it tries to do.  There are 

 5    some caveats, but, yes, that is what it is trying to do.

 6         Q.    Did you hear Dr. Fisher testify that on the basis 

 7    of the Bureau's loss function analysis, adjustment would 

 8    make a big improvement in distributional accuracy?

 9         A.    I did.

10         Q.    Do you agree with his analysis?  

11               MR. SOLOMON:  I object.  

12               THE COURT:  Overruled.

13         A.    I do not.

14         Q.    Why not?

15         A.    What he pointed out is, I think true, that the 

16    estimated risk from the census is maybe ten times the 

17    estimated risk from adjustment.  But that is only part of 

18    the story.  The Bureau was looking at differences, and I 

19    think it is reasonable to look at differences.  

20               You can think about how much improvement in 

21    accuracy is being suggested by these loss function analyses 

22    on the average over the 51 areas.  I made such calculation 

23    based on one table in P-16.  I don't think the other tables 

24    came out very differently, although I didn't do them.  

25               The improvement in accuracy from adjustment using 
                                                              2424

 1    the squared error criterion on the average over the 51 

 2    areas, taking square roots to get back to the units we are 

 3    interested in, percentage points, the expected improvement 

 4    in accuracy is just under four one-hundredths of a 

 5    percentage point.

 6         Q.    Did you make the same kind of computation using 

 7    an absolute error criterion?

 8         A.    I did, and the answer is one one-hundredth of a 

 9    percentage point.

10         Q.    What is one one-hundredth?

11         A.    The improvement in accuracy.

12         Q.    Why are the numbers so small?

13         A.    The change in shares for the various states are 

14    really rather small.  I think the biggest one is California, 

15    at two-tenths of a percentage point.  That is a small 

16    number.  The other ones are even smaller.  Some of the 

17    changes suggested by adjustment are going to be errors.  So 

18    the improvement in accuracy expected from adjustment is, I 

19    think, really rather small.

20         Q.    Dr. Freedman, could you turn to Defendants' 

21    Exhibit 64.  I would like you to turn particularly to page 

22    19 and look at Table 9.

23         A.    Yes.

24         Q.    What data are shown in this table?

25         A.    Table 9 shows my replication of the Bureau's loss 
                                                              2425

 1    function analysis using one of their particular ways of 

 2    estimating state level biases in the dual-system estimator.

 3         Q.    Which method of allocation did you use?

 4         A.    There are two of them.  One of themties called 

 5    PRODSE, P R O D S E.  That is the one used in Table 9.  The 

 6    other one is even less pronounceable.  Ken Wachter and I 

 7    call it UNWEPRS, U N W P E R S.  This Table 9 is based on 

 8    PRODSE.

 9         Q.    According to Table 9, how are the gains from 

10    adjustment distributed geographically?

11         A.    What Table 9 shows -- 

12               THE COURT:  What page is Table 9 on?  

13               MR. SITCOV:  That is on 19, your Honor.  

14               THE COURT:  Thank you.  

15               MR. SOLOMON:  And we don't have the program for 

16    this either, your Honor.  

17               THE COURT:  I knew you were going to say that.  

18               MR. SOLOMON:  To put a point on it, if you look 

19    at the Bureau document which purports to reflect here Table 

20    6, and you look at the line that says "Risk from 

21    Adjustment," the column, you get a number about half of what 

22    Professor Freedman shows.  He has 88.  The Bureau document 

23    has 48.  

24               THE COURT:  Perhaps he can enlighten us on that.

25         Q.    Can you enlighten us on that?
                                                              2426

 1         A.    I can try.  I don't know if this is going to go 

 2    any better than the absolute value and the squares.  

 3               The Bureau's method for doing the loss function 

 4    analysis uses what is called a Monte Carlo technique or a 

 5    bootstrap technique.  They generate a thousand targets by a 

 6    certain random process, which I could explain if the Court 

 7    wished, and they generate a thousand simulated dual-system 

 8    estimators by another process which I could, given time, 

 9    also explain.  Then they do their calculations based on 

10    those simulated data.  

11               The approach I took in Table 9 is quite 

12    different.  What I did in Table 9 is an analytical 

13    calculation which in principle gives exact answers.  It is 

14    not subject to the vagary of these simulations.  Counsel is 

15    quite right, there is a difference in the risk from 

16    adjustment that I computed at the bottom of Table 9, which I 

17    got as 88, and the number which the Bureau gives.  

18               The Bureau's number is computed by that Monte 

19    Carlo process that I described, and I believe there is a 

20    small mistake in their Monte Carlo work.  I have discussed 

21    that with the programmer at the Bureau, Freddie Navarro.  We 

22    have kind of checked each other's calculations.  He has more 

23    or less agreed with the principal ingredients for my 

24    calculation.  I don't want to put words in his mouth, 

25    especially since he is not here -- 
                                                              2427

 1               THE COURT:  That's the best time to do it.

 2         A.    I think it would be fair to say that I 

 3    acknowledges that one of the numbers that he got from his 

 4    Monte Carlo is somewhat on the low side.  This is all 

 5    discussed on pages 32 and 33 of my report.  There is a 

 6    little table which discusses the discrepancy between my 

 7    number for the risk from adjustment and the Bureau's number.  

 8               MR. SOLOMON:  Your Honor, I am going to move to 

 9    strike, just for the record.  But my point is more important 

10    than that.  

11               This witness, having the data and the programs 

12    from the Bureau, can say, I think they made a mistake.  

13    Well, I think the Professor made a mistake, but I can't 

14    check it because I don't have his program.  

15               THE COURT:  I understand that, for about the 

16    fortieth time.  I am going to permit it.  But I will strike 

17    the last answer as blatant hearsay.  What Freddie said is 

18    interesting, but hearsay nonetheless.  

19    BY MR. SITCOV:

20         Q.    Forget about Freddie now.  

21               I believe I asked you if according to Table 9 you 

22    gained some adjustment -- well, I don't know if I did.  I 

23    will ask you now.  The gains from adjustment, how are they 

24    distributed geographically?

25         A.    I think Table 9 shows a high degree of 
                                                              2428

 1    geographical concentration in the gains from adjustment.  By 

 2    my calculation, the total estimated risk difference is 667.  

 3    I am using parts per hundred million because the numbers are 

 4    so small.  So this gets them to where you can see them.  

 5               The total gain from adjustment to risk difference 

 6    in favor of adjustment is 667 parts per million.  That is 

 7    the bottom line in the risk difference column.  California 

 8    contributes 486 parts per million.  So California is really 

 9    a very big contributor to this risk difference total.  

10               There are some other states which make much 

11    smaller but still relatively large contributions, like 

12    Pennsylvania comes in at 61.  But most of the states are 

13    putting very small numbers in that risk difference column.

14         Q.    Could you turn to page 21 of that report, please.  

15    Do you see that Figure 2?

16         A.    I do.

17         Q.    What data are shown in Figure 2? 

18         A.    Figure 2 just plots the data from Table 9.  The 

19    vertical line represents an estimated risk difference of 

20    zero.  The horizontal bars show the estimated risk 

21    differences for the several states and D.C.  California 

22    comes in with that giant bar at the top.  Making the 

23    assumptions that the Bureau made going into this table, it 

24    surely looks like California is being helped by adjustment.  

25    New York at the bottom is being hurt by adjustment.  When I 
                                                              2429

 1    say hurt, I mean in terms of accuracy.  And in between is in 

 2    between.

 3         Q.    When you say "in between is in between," why it 

 4    doesn't appear as though there are any marks for most of the 

 5    states.  Why is that?

 6         A.    For most of the states the gains from adjustment 

 7    are so small that when you plot them on this figure, their 

 8    bars just blur into the vertical line.

 9         Q.    How would you describe California's effect on the 

10    results shown in this table?

11         A.    California, I think, is the dominant source of 

12    the apparent gain from adjustment, apparent gain in accuracy 

13    from adjustment.

14         Q.    In your opinion, is a loss function a good way to 

15    summarize the gains in accuracy from adjustment?

16         A.    No, I don't believe it is.  I think that when you 

17    have one large source of gain like California in Figure 2 on 

18    page 21, adding up the gains and losses is not a good way to 

19    summarize these data.  It seems to me we have a situation 

20    here with one or two big outliers, as we call them.  You 

21    just don't summarize data like that by adding up the 

22    numbers, at least not without looking at them and thinking 

23    about them.

24         Q.    Do you recall hearing Dr. Fisher testify about 

25    whether California was running away with adjustment on the 
                                                              2430

 1    basis of his data?

 2         A.    I do.

 3         Q.    Do you agree with his testimony?

 4         A.    I don't.

 5         Q.    Why not?

 6         A.    As I understood him, he was saying that he didn't 

 7    think a big number like California's number in a sum like 

 8    this would be skewing the results.  But as I understood his 

 9    testimony, he was somehow thinking that we had laid down, 

10    the Bureau had laid down or the Secretary of Commerce or 

11    somebody had laid down the decision rule where we were going 

12    to use squared error or absolute error and add up all those 

13    reports.  If you made up a kind of hard and fast commitment 

14    to do that as a way to make your decision, then it probably 

15    isn't the right thing to do afterwards to start breaking 

16    down the sum.  

17               But if you are using a loss function analysis as 

18    a way to summarize data, as a way to summarize the gains and 

19    losses, which is the way I would think about it and I think 

20    the way the Bureau was thinking about it, then you have to 

21    ask yourself, is this a good summary statistic?  The answer 

22    to that question is no, it is not a good summary statistic 

23    because California is skewing the results.

24         Q.    Could you turn to page 23 of Exhibit 64.

25         A.    Yes.
                                                              2431

 1         Q.    Do you see Table 10?

 2         A.    Yes.

 3         Q.    What does that table show?

 4         A.    The bottom line of that table just shows the 

 5    totals from Table 9.  It shows the estimated risk from the 

 6    census, 755 parts per hundred million; estimated risk from 

 7    adjustment, 88 on the same scale, a difference of 67.  It 

 8    also puts a standard error on the 667.  The standard error I 

 9    compute as 253 parts per hundred million.  

10               These calculations are all based on the 

11    post-enumeration survey.  That sample data.  As a result 

12    this estimated risk difference, as the name suggests, is 

13    only an estimate.  It is subject to sampling error.  We can 

14    compute the size of that sampling error in terms of the 

15    standard error.  

16               I did that.  I think I did that on quite a 

17    conservative basis, which, granted all the assumptions that 

18    the Bureau was making going into this analysis.  The answer, 

19    the standard error is 253.  So that 667 incorporates within 

20    itself some unknown amount of sampling error, and the 253 

21    gives you a kind of ballpark estimate as to the amount of 

22    that sampling error.  

23               The other lines on that table make a separate 

24    analysis for the biggest contributor, California, which 

25    comes in with a risk difference, an estimated risk 
                                                              2432

 1    difference of 486 plus or minus 218.  California was what I 

 2    call a triple-digit contributor to the risk difference.  

 3               There were several double-digit contributetors, 

 4    Massachusetts, Ohio, Texas, and Pennsylvania.  The estimated 

 5    risk difference for them is 135 with a standard error of 58.  

 6               So those risk differences seem to me to be 

 7    reasonably well estimated.  It would be kind of hard to 

 8    explain them on the basis of sampling error, given 

 9    everything that we are given here.  

10               But for the other 46 states -- sorry -- the other 

11    45 states and D.C., the estimated risk difference is 46 

12    parts per hundred million with a standard error of 45.  That 

13    is just at the chance level.  In other words, for the other 

14    45 states and D.C., you can't tell whether adjustment is 

15    helping or hurting, are we going north or south.  You cannot 

16    tell.

17         Q.    In your opinion, is Table 10 evidence in favor of 

18    adjustment?

19         A.    It cuts both ways.  The total line, 667, is 

20    evidence for adjustment.  The standard error shows that that 

21    risk difference is estimated stablely enough so that you 

22    should pay some attention to it.  That is pro adjustment.  

23               The other side, however, of the argument is that 

24    for 45 states and D.C., the data are not telling you that 

25    adjustment is doing anything for you in terms of improving 
                                                              2433

 1    accuracy.  So there is some conflict in the evidence on this 

 2    table.

 3         Q.    Could you turn to page 46 of that report.

 4         A.    Yes.

 5         Q.    Do you see Dr. Table B-1?

 6         A.    Yes.

 7         Q.    What does this table show?

 8         A.    If we could look back at Table 10 for just one 

 9    second, at the bottom of Table 10 there is an estimated risk 

10    difference for it is whole country, for all 51 areas, of 

11    667, standard error of 253.  I am asking the question 

12    whether you could get such a big estimated risk difference 

13    just due to chance even if the real risk difference was zero 

14    so that adjustment was not improving on the census in terms 

15    of distributive accuracy.  

16               As I said a little bit ago in another context, 

17    the way we statisticians do that is we would take the 

18    estimated risk difference, 667, and divide by the standard 

19    error, 253, to get a T statistic.  That T statistic of 2.64 

20    is shown at the bottom of Table B-1 in the column headed 

21    "Bureau."  

22               There is also a T statistic for California and a 

23    T statistic for the double-digit contributors, 

24    Massachusetts, Ohio, Texas, and Pennsylvania.  There is also 

25    a T statistic for the other 45 states and D.C. 
                                                              2434

 1               What that first column shows, I think, is that 

 2    for California and for that group of the other four states, 

 3    there is some evidence, it is not overwhelming but there is 

 4    reasonable evidence that adjustment will improve on 

 5    distributive accuracy.  For the country as a whole there is 

 6    an improvement.  

 7               But if you focus on the 45 states and D.C., other 

 8    than California, Massachusetts, Ohio, Texas, and 

 9    Pennsylvania, we are operating at the chance level here.  A 

10    T statistic of 1 indicates that apparent estimated risk 

11    difference could be arising just due to sampling error.

12         Q.    Dr. Freedman, could you take a look at 

13    Plaintiffs' Exhibit 698.  Do you have that?

14         A.    I do.  

15               THE COURT:  I take it you are not going to finish 

16    today?  

17               MR. SITCOV:  I wouldn't take I that.  I can 

18    probably finish in the next 20 minutes or so, or the next 

19    half hour.  

20               THE COURT:  Do it.

21         Q.    Do you have Plaintiff's Exhibit 698 in front of 

22    you?

23         A.    Yes.

24         Q.    What does that exhibit show?

25         A.    I believe that it is a calculation supervised by 
                                                              2435

 1    Dr. Fisher of the same T statistic that is computed in the 

 2    first column at the bottom of Table B-1.  I am getting a T 

 3    statistic of 2.64, and he is getting 5.68.

 4         Q.    That is Table B-1 on page 46 of Defendants' 

 5    Exhibit 64?

 6         A.    It is.

 7         Q.    Can you explain the difference between your T 

 8    statistic and his?

 9         A.    I think so.  I think mine is right, and I think 

10    his is wrong.  I heard him testify as to how that number was 

11    computed.  By two slightly different methods, I have 

12    succeeded in following his recipe and getting that number.  

13    I think there is a serious theoretical mistake built into 

14    his calculation.

15         Q.    What is that theoretical mistake?

16         A.    I think that is the same theoretical mistake that 

17    is built into the Bureau's preliminary loss function 

18    analysis.  As I understood him, he is considering 

19    variability in the targets but he is not considering 

20    variability in the production dual-system estimator, which 

21    is the main driving or one of the main driving things which 

22    might make adjustment worse.  So I think in his calculation 

23    he has ignored the major source of variance that should be 

24    paid attention to.

25         Q.    Did the Bureau make a change in its loss function 
                                                              2436

 1    analysis after it made that mistake?

 2         A.    It did.

 3         Q.    Turning back again to Table B-1 on page 46, what 

 4    does the second column in that table show, the one under 

 5    "Alternative"?

 6         A.    One main ingredient in the loss function analysis 

 7    is the variance/covariance matrix for the smoothed 

 8    adjustment factors.  The Bureau computes it on the basis of 

 9    its assumption on that righthand chart over there.  The 

10    middle column computes it on the basis of my scissors and 

11    scotch tape modification of that assumption, which is on the 

12    lefthand chart.  

13               So the middle column is based on my alternative 

14    assumption about the variance/covariance matrix for the 

15    errors in the second equation of the production smoothing 

16    model, which cranks through to different variance/covariance 

17    matrix for the smoothed adjustment factors, which in turn 

18    gets fed into the loss function analysis.

19         Q.    What does B-1 suggest to you, Table B-1?

20         A.    Staying for a moment with that middle column, we 

21    now have a negative loss in the other 45 states and D.C.  In 

22    other words, adjustment is estimated to be harming the 

23    accuracy for the aggregate of those 45 states and D.C., but 

24    it is still helping in California and in the other four 

25    double-digit states.  But all the numbers in that column, 
                                                              2437

 1    including the national total, are at the chance level.  The 

 2    T statistic is 1.44.  

 3               The third column considers a different 

 4    modification to the variance/covariance matrix, where I just 

 5    doubled it.  That is the variance/covariance matrix for the 

 6    smoothed adjustment factors.  So you can just double that 

 7    matrix and then feed it into the loss function analysis.  

 8               Again, the national total is not really 

 9    significant.  There is a tiny negative number for the other 

10    45 states and D.C., indicating an estimated loss in accuracy 

11    from adjustment but at the chance level, and the numbers for 

12    California and the four double-digit states are 1.45 and 

13    1.35, which are not statistically significant.

14         Q.    Why did you double the variance?

15         A.    As I said earlier, I believe that the 

16    variance/covariance matrix computed from the Bureau's 

17    smoothing model substantially understates the amount of 

18    sampling error in those smoothed adjustment factors.  The 

19    Undercount Steering Committee, I thought, in their report 

20    expressed the same opinion.  They had a range of 1.7 to 3.  

21    I picked 2 as a factor consistent with what I saw in some 

22    work that is reported here and which was a nice round number 

23    in the middle of their range.  

24               I must say that if we modified that 

25    variance/covariance matrix to take into account other 
                                                              2438

 1    sources of error besides sampling error, I think that 

 2    multiplication by 2 is a very conservative adjustment.

 3         Q.    What does the final column show?

 4         A.    The final column shows the same kind of analysis 

 5    for the raw adjustment factors.  The results are similar, 

 6    although you get quite a significant estimated expected loss 

 7    from adjustment in the other 45 states and D.C.

 8         Q.    Do the results that are displayed in Table B-1 

 9    counsel in favor of adjustment?

10         A.    On the whole, I think they counsel against 

11    adjustment.  If you use the original Bureau 

12    variance/covariance matrix for the smoothed adjustment 

13    factors, there is some evidence of good work by adjustment, 

14    good improvement in accuracy, and for many states the 

15    apparent improvement in accuracy is at the level you would 

16    expect by chance. 

17               If you pass to different and I think either less 

18    unrealistic or more unrealistic, depending on how you feel, 

19    assumptions about variances, the significance of the 

20    estimated expected risk just goes away, I think.

21         Q.    Does loss function analysis take into account the 

22    problem of heterogeneity?

23         A.    That is another question without a simple answer.  

24               THE COURT:  Without?

25               THE WITNESS:  Without a simple answer.  
                                                              2439

 1         A.    Heterogeneity comes into the story three times.  

 2    It comes in when you calculate the raw variances using the 

 3    jackknife.  That technique does look at block-to-block 

 4    variability.  That block-to-block variability does reflect, 

 5    in part, heterogeneity.  So it is there.  

 6               It also comes into correlation bias.  That is, 

 7    the heterogeneity in capture probabilities or recapture 

 8    probabilities is, I think, what correlation bias is all 

 9    about.  The Bureau, for better or worse, has made estimates 

10    of correlation bias, which get fed into this loss function 

11    analysis.  

12               However, there is a third place that 

13    heterogeneity should come in but does not.  When you pass 

14    the adjustment down from poststrata to smaller areas, even 

15    states, you are relying on the homogeneity assumption.  I 

16    think that assumption is wrong, and I think that the errors 

17    that are introduced by heterogeneity even when you adjust 

18    states, are not trivial.  I think that Dr. Ken Wachter 

19    testified about that.  

20               At any rate, the Bureau's loss function analysis 

21    just ignores those errors.

22         Q.    Does the smoothing process introduce bias?

23         A.    I believe it does.  It seems to me that we heard 

24    or I heard testimony agreeing with that from many of the 

25    plaintiffs' experts, so that I think is one point on which, 
                                                              2440

 1    among the experts on the different sides of this case, there 

 2    is some degree of agreement.

 3         Q.    Does the loss function analysis the Bureau did 

 4    take that bias into account?

 5         A.    I don't believe it does, and it is actually worse 

 6    than that.  The bias in introduced by smoothing would, I 

 7    think, be passed through to these targets, this estimate of 

 8    truth.  So the census would actually be charged for that 

 9    bias as an error, and I think adjustment would be credited 

10    with it.

11         Q.    Does the loss function analysis depend on 

12    estimates of bias in the state shares from the dual-system 

13    estimates?

14         A.    Indeed it does.

15         Q.    Does the total error model measure bias in 

16    estimated state shares?

17         A.    No, it does not.  The total error model estimates 

18    bias, various kinds of bias, for 13 evaluation poststrata.

19         Q.    So how did the Bureau get the estimated biases 

20    for the states?

21         A.    It does that by an allocation procedure, a 

22    modeling procedure.  I did mention earlier that there were 

23    two different ones used, PRODSE and UNWEPRS.  

24               You start with the 13 evaluation poststrata.  

25    Each evaluation poststratum breaks down into 12 age/sex 
                                                              2441

 1    groups.  There is a procedure which I do not regard as so 

 2    reasonable for sharing the biases down to the 12 age/sex 

 3    groups within each evaluation poststratum.  We now have 156 

 4    different groups, 12 times 13.  

 5               The Bureau then passes the bias down from each 

 6    age/sex group within an evaluation poststratum to the 

 7    various individual poststrata that make it up.  That is 

 8    where the differences between PRODSE and UNWEPRS come in.  

 9    They have two different ways of doing that. 

10         Q.    Do you remember Dr. Fisher testifying that the 

11    two different methods, PRODSE and UNWEPRS, were fairly 

12    different methods of allocating bias?

13         A.    I do.

14         Q.    In your opinion, was Dr. Fisher correct?

15         A.    I don't agree with him on that point either.  It 

16    seems to me that the two methods are identical at the level 

17    of the 156 age/sex groups.  There is some difference in the 

18    way they get down from the age/sex group to the component 

19    poststrata, but there is also a strong degree of similarity.  

20               One allocates proportional to the census count 

21    for the poststratum or the adjusted census count, and the 

22    other one allocates proportional to the unweighted P sample 

23    size.  Those numbers have some considerable degree of 

24    correlation.  

25               Most important, neither method permits geographic 
                                                              2442

 1    variation in bias across states.  The variation is all 

 2    within poststrata.

 3         Q.    Did you hear Dr. Fisher testify about the 

 4    robustness of the loss function analysis?

 5         A.    I did.

 6         Q.    Do you recall what his opinion was?

 7         A.    I thought he testified that the loss function 

 8    analysis was robust in this respect.

 9         Q.    In your opinion, is the loss function analysis 

10    robust in this respect?

11         A.    No, I don't believe it is.  If you change the 

12    method for allocating bias, for example, make it 

13    proportional to the undercount, and change the numbers 

14    slightly, you can get quite different results.  

15               I did one little computer experiment like that 

16    where I think I made the bias, I set the bias to one-third 

17    of the undercount and kept the variance/covariance matrix at 

18    double the Bureau's.  You get a negative estimated expected 

19    risk for the whole country, and it is not statistically 

20    significant.  

21               So by making those changes you moved from an 

22    estimated expected risk which is positive and quite 

23    significant to an estimated expected risk which is negative 

24    and insignificant.  That seems to me to be quite a serious 

25    change obtained by changing some of the assumptions in the 
                                                              2443

 1    loss function analysis.

 2         Q.    Dr. Freedman, could you turn to Plaintiffs' 

 3    Exhibit 686.  Do you have that in front of you, Dr. Fisher?  

 4    Excuse me.  Dr. Freedman.  You don't look alike.

 5         A.    I have it.

 6         Q.    Do you recall Dr. Fisher testifying with that 

 7    chart?

 8         A.    I do.

 9         Q.    Do you recall that he testified that that chart 

10    suggested or was evidence in favor of adjustment?

11         A.    Yes.

12         Q.    In your opinion, is the chart evidence in favor 

13    of adjustment?

14         A.    I think it would be if I believed the numbers, 

15    but I don't think I do.

16         Q.    Why don't you believe them?

17         A.    The procedure he described for computing these 

18    probabilities I think is really quite a biased one.  Even if 

19    a case where the census was perfect, not that I am saying it 

20    is, but just hypothetically to make the point, in a case 

21    where the census was perfect so that adjustment would be 

22    making all the state shares worse by some amount or another, 

23    I think his procedure will suggest that approximately half 

24    the states are expected to be improved.  So I think this way 

25    of calculating probabilities is quite a biased way to make 
                                                              2444

 1    estimates.

 2         Q.    Is there some particular mistake that he made?

 3         A.    I think again it comes down to ignoring in a 

 4    certain part of the calculation the variability in the 

 5    dual-system estimator.

 6         Q.    Dr. Freedman, have you considered the number of 

 7    states whose population shares would be made better by 

 8    adjustment?

 9         A.    I have.

10         Q.    How did you do that?

11         A.    I used a technique which is quite standard in the 

12    field of multiple comparisons.  It is a technique called 

13    simultaneous confidence intervals.  It is almost, but not 

14    quite, a hypothesis test in the classical sense.

15         Q.    Why did you make that test 

16               THE COURT:  Why?  

17               MR. SITCOV:  Yes, why.

18         A.    First of all, there seemed to be, in the report 

19    of the Undercount Steering Committee, the report of the 

20    director, and the Secretary's decision document, 

21    considerable interest in the number of states such that the 

22    accuracy of their population shares was being improved by 

23    adjustment.  That was one motivation.  

24               Another motivation is that the guidelines 

25    strongly suggest hypothesis testing techniques, at least to 
                                                              2445

 1    me.  I think the Secretary explicitly called for a 

 2    hypothesis test, and I think Under Secretary Darby called 

 3    for a hypothesis test.  

 4               I do think that is an appropriate way of getting 

 5    at the question of whether the data are pushing us to think 

 6    that adjustment will improve the accuracy of population 

 7    shares.

 8         Q.    Was that an exercise in trying to count the 

 9    number of states whose signs went positive or negative?

10         A.    No, it was not.  That is a point on which Dr. 

11    Fisher and I agree.  Sign counting like that does give you 

12    some information, but I don't think it is such a good way 

13    and I think neither does he, if I understood him correctly, 

14    to estimate the numbers we are interested in.

15         Q.    Could you turn to page 39 of Defendants' Exhibit 

16    64.  Do you have it there?

17         A.    Yes.

18         Q.    Do you see Table 17?

19         A.    Yes.

20         Q.    What does that table represent?

21         A.    That table represents or summarizes the 

22    calculations that I have been describing.  You can see at 

23    the left various hypotheses are put forward.  The middle 

24    column of the table explains the basis for the calculations 

25    and what assumptions are being made about the variances and 
                                                              2446

 1    the biases.  The last three columns tell you the results of 

 2    the procedure.  

 3               What these data suggest is that there is very 

 4    little evidence in the data pushing you to think that 

 5    adjustment will be making the shares more accurate for a 

 6    large number of states with a large amount of population.  

 7    It is quite consistent with the data, for example, to think 

 8    that -- I will just pick one of these things -- does 

 9    adjustment make shares less accurate for 35 states with 57 

10    percent of the population?  That seems to me to be 

11    reasonably consistent with the data.  

12               So what this table suggests to me is that we do 

13    not have strong evidence in favor of the proposition that 

14    adjustment is making shares more accurate for lots of states 

15    with lots of people.  It may be helping somewhere, probably 

16    is, but not as a general proposition.

17         Q.    Dr. Freedman, if I understand you correctly, you 

18    either were in court for the testimony of all of plaintiffs' 

19    witnesses or you read the deposition, is that right?

20         A.    I read the transcript for those --

21         Q.    I'm sorry, transcripts.

22         A.    -- portions of the testimony I missed.

23         Q.    As I understand it, you crunched a lot of numbers 

24    yourself and you have considered a lot of data that the 

25    Bureau has generated, is that correct?
                                                              2447

 1         A.    I have crunched more numbers than I want to think 

 2    about.

 3         Q.    What, in your opinion, is the bottom line on 

 4    whether or not the 1990 Decennial Census should be adjusted 

 5    in the manner the plaintiffs are advocating?

 6         A.    I would advise against adjustment.  It seems to 

 7    me there is no strong evidence on the table to show that 

 8    adjustment will improve the distributional accuracy for 

 9    states or other areas, and I think there is a real risk that 

10    adjustment would actually put in more error than it took 

11    out.  

12               MR. SITCOV:  No further questions, your Honor.  

13               THE COURT:  Good.  You kept your word.  

14               We will recess for the day.  We will resume 

15    tomorrow morning with cross-examination at 9:30.  

16               (Adjourned to 9:30 a.m., May 28, 1992) 

17    

18

19

20

21

22

23

24

25
