                                                              2449

 1    UNITED STATES DISTRICT COURT
      EASTERN DISTRICT OF NEW YORK
 2    ------------------------------x
      THE CITY OF NEW YORK, et al.,
 3    
                         Plaintiffs,          88 Civ. 3474 (JMcL)
 4    
                 v.                           
 5    
      UNITED STATES DEPARTMENT OF COMMERCE,
 6    et al., 
      
 7                       Defendants.
      ---------------------------------x
 8    CITY OF ATLANTA, et al.,
                     
 9                       Plaintiffs,
      
10                v.                          92 Civ. 1566 (JMcL)
                                              
11    MOSBACHER, et al.,
      
12                       Defendants.
      ---------------------------------x
13    FLORIDA HOUSE OF REPRESENTATIVES, 
      et al.
14    
                         Plaintiffs,
15                v.                          92 Civ. 2037 (JMcL)
      
16    BARBARA H. FRANKLIN, Secretary of
      Commerce,
17    
                         Defendant.
18    --------------------------------x
      
      
19    
                                              May 28, 1992
20                                            9:30 a.m.
      
21    
      
22    Before:
      
23               HON. JOSEPH M. McLAUGHLIN,
      
24                                            Circuit Judge
      
25
                                                              2450

 1               THE COURT:  Mr. Sitcov, you look poised to 

 2    strike.  

 3               MR. SITCOV:  That is just how I feel, your Honor.  

 4    I neglected to offer into evidence a couple of exhibits that 

 5    Dr. Freedman testified to yesterday and I want to make that 

 6    offer now.  They are Defendants' Exhibit 66 -- 

 7               THE COURT:  Why don't you take them in order.  I 

 8    think there were some earlier numbers.  

 9               MR. SITCOV:  Yes, there were.  The earlier 

10    numbers are 62, 63, and 64.  Those are the reports that Dr. 

11    Freedman prepared that he testified about yesterday.  66 is 

12    a letter with enclosures that Dr. Freedman testified about, 

13    the letter that Dr. Freedman received from the Census 

14    Bureau, addressed to him, and signed by a Census Bureau 

15    official.  Enclosure 3 was the smoothing model that he 

16    testified about yesterday.  

17               Then Exhibit 68 was Dr. Freedman's modification 

18    to the second assumption in the smoothing model.  

19               THE COURT:  That's it?  

20               MR. SITCOV:  That's it, yes.  

21               THE COURT:  Mr. Solomon?  

22               MR. SOLOMON:  Good morning, your Honor.  

23               THE COURT:  Good morning.  

24               MR. SOLOMON:  We don't object to 66, which is the 

25    letter.  
                                                              2451

 1               THE COURT:  66 is admitted. 

 2               (Defendants' Exhibit 66 for identification was 

 3    received in evidence) 

 4               MR. SOLOMON:  Exhibit 68, it is a one-page 

 5    re-mark of a -- 

 6               THE COURT:  That was the one we had in the 

 7    charts, yes.  

 8               MR. SOLOMON:  Right.  

 9               MR. SITCOV:  That is the one with V sub 3.  

10               THE COURT:  Yes, V sub 3, I remember it well.  

11               MR. SOLOMON:  It is one altered page of a report.  

12    We don't have any objection to its coming in for 

13    illustrative purposes, but I don't think there is anything 

14    else to it.  I don't think it is otherwise evidentiary.  

15               THE COURT:  We have enough evidentiary testimony 

16    about it in the witness, so I will take it in as an adjunct 

17    to his testimony, illustrative only, 68. 

18               (Defendants' Exhibit 68 for identification was 

19    received in evidence) 

20               MR. SOLOMON:  With respect to 62, 63, and 64, we 

21    have the same objections that we did yesterday and then 

22    some.  Mr. Sitcov got through his examination and pointed 

23    the witness to what he believed were the relevant parts 

24    without having these documents in evidence.  The parts that 

25    weren't discussed really do contain not just double and 
                                                              2452

 1    triple hearsay -- 

 2               THE COURT:  Your problem is with the stuff that 

 3    was never discussed?  

 4               MR. SOLOMON:  That is correct, your Honor.  

 5               THE COURT:  I agree with you.  I will take those 

 6    parts of the documents 62, 63, and 64 which contain charts 

 7    and diagrams and things about which the witness testified.  

 8    All of the rest I will exclude. 

 9               (Defendants' Exhibits 62, 63 and 64 for 

10    identification werr received in evidence)

11               THE COURT:  You are ready to resume, Mr. Solomon, 

12    or to start?  

13               MR. SOLOMON:  Ready to start.  

14               MR. RIFKIND:  It just feels that way.  

15    DAVID FREEDMAN      resumed
                                 

16    CROSS-EXAMINATION 

17    BY MR. SOLOMON:

18         Q.    Good morning, Professor Freedman.  A very 

19    important part of the testimony that you gave yesterday 

20    concerned your discussion of the standard errors surrounding 

21    the undercount estimates in your opinion, is that correct?

22         A.    That was one major topic, yes.

23         Q.    I would like you to look at DX-64, which is the 

24    paper on loss functions, page 4, Table 2.  

25               When we were on this chart yesterday, I think in 
                                                              2453

 1    one of my few objections I observed that it didn't have the 

 2    standard error information.  I believe you said, on page 

 3    2372, "They have been given the data.  I gave it to them 

 4    myself.  It's on that diskette."  

 5               I would like to show you a copy of the letter 

 6    that was given to us when we got the diskette.  

 7               MR. SOLOMON:  May I, your Honor?  

 8               THE COURT:  Yes.

 9         Q.    We have marked that for identification as PX-766.  

10               I believe you testified that the standard errors 

11    here were computed using variance/covariance matrices, 

12    correct?

13         A.    Yes.

14         Q.    Would you please identify for us on this list of 

15    the files that you gave us which variance/covariance matrix 

16    you used in the calculation of the standard errors that 

17    appear on page 4, Table 2?

18         A.    The major ingredient for those standard errors is 

19    the Bureau's final covariance matrix for the smoothed 

20    adjustment factors which appear at disks 1 through 10 dot 

21    COV on the last page.  COV, C O V.  You also need the 

22    evaluation -- sorry -- you need the adjustment factors 

23    themselves, which are in the SMTH1392.DAT file, and you need 

24    the population counts for the various states.  Those appear 

25    earlier as state dash two digits dot DAT.  
                                                              2454

 1               There is yet another file which I think is called 

 2    STRESID dot DAT.  These, by the way, are also files that are 

 3    listed with different names in Dr. Fisher's exhibits.  Not 

 4    the covariances, but the population files.

 5         Q.    Let's stick with the covariances, because I think 

 6    that is what you told me yesterday I could look at to derive 

 7    these standard errors.  The covariance matrices that are in 

 8    the files that you gave us are covariances for the 

 9    adjustment factors, not for the shares, correct?

10         A.    That's right.

11         Q.    You created a separate covariance/variance matrix 

12    in order to calculate the standard errors that appear on 

13    page 4, Table 2, correct?

14         A.    Right.

15         Q.    You didn't give us that covariance matrix, did 

16    you?

17         A.    I did not.

18         Q.    What formula did you use to calculate the 

19    covariance/variance matrix that you didn't give us but that 

20    you needed to prepare these standard errors?

21         A.    You, first of all, derive a variance/covariance 

22    matrix for the adjustments to the states.  Then you derive 

23    by what is called the delta method the variance/covariance 

24    matrix for the shares.

25         Q.    Is the delta method the only method that could be 
                                                              2455

 1    used to calculate a variance/covariance matrix?

 2         A.    I don't know that it is the only one.  I would 

 3    think it is the standard one to get from the numbers to the 

 4    shares.

 5         Q.    What was given to us you didn't tell us that you 

 6    used the delta method, did you?

 7         A.    May I look at my report for a second?

 8         Q.    Of course.

 9         A.    Would you please turn to page 31.  There is an 

10    equation 9 on that page.  Below that equation on the page 

11    there is a block of type which refers to G.  That is the 

12    variance/covariance matrix that I believe you were referring 

13    to that I had computed from the Bureau's data.  It says that 

14    I computed X, those are the adjustments, from the Bureau's 

15    smoothed adjustment factors, and G from the Bureau's 

16    covariance matrix for these factors -- that would be the  

17    stuff in the dot COV files -- via the delta method.

18         Q.    Here you are telling us how you computed the 

19    variance/covariance matrix for Table 9, not for Table 2, 

20    correct?

21         A.    It is the same matrix and the same method.

22         Q.    Where did you tell us that, sir?

23         A.    I don't know that I did in the report.

24         Q.    When you calculated the variance/covariance 

25    matrix --
                                                              2456

 1         A.    Forgive me.  If I may continue.  There is a 

 2    parenthesis in there which explains that the Bureau provided 

 3    state level census counts by poststrata.  Those are the 

 4    state dash two-digit files.  And counts for the residual 

 5    population, that is the STRESID file.

 6         Q.    I just want to focus on what you told me to look 

 7    at yesterday, and that is the variance/covariance matrix.  

 8    We have now established that we didn't have what you told me 

 9    to look at.  

10               MR. SITCOV:  I object.  

11               THE COURT:  I will sustain the objection.

12         Q.    I want the record to be perfectly clear.  You did 

13    not give us the variance/covariance matrix which you 

14    calculated and which you needed in order to calculate the 

15    standard errors on Table 2, isn't that right?

16         A.    I did not give you the matrix G, that's correct.

17         Q.    Did you, in calculating the matrix G, utilize the 

18    share data that you had or did you do it analytically based 

19    on the Bureau's numbers?

20         A.    I may not be answering your question.  Let me 

21    try.  If I don't, just -- 

22               THE COURT:  He will tell you.  

23               THE WITNESS:  I actually wasn't afraid he 

24    wouldn't, your Honor.

25         A.    I recomputed the shares myself using the smoothed 
                                                              2457

 1    adjustment factors that the Bureau provided.  I did not use 

 2    the published shares.

 3         Q.    There are a number of other tables that you 

 4    looked at in this report yesterday, including Table 9, which 

 5    have plus or minus numbers, risk differences, you also do it 

 6    on Table 10.  We couldn't calculate any of those based on 

 7    the information that was on these files, isn't that correct?

 8         A.    Oh, I think that is totally wrong.  I computed 

 9    those numbers from the data in these files, and I tried hard 

10    to explain how I did it in these technical pages in the 

11    report.  If I failed, forgive me, but I did try.

12         Q.    The fact of the matter is, unless you tell us 

13    that you are using the delta method for calculating the 

14    variance/covariance matrix, we would have to guess to know 

15    which one you used, correct?  

16               THE COURT:  Except that he said that was the 

17    standard method.

18         Q.    There is more than one standard method, isn't 

19    there?

20         A.    For computing the covariance matrix for shares?

21         Q.    Yes.

22         A.    I suppose there is.  I find it hard to think what 

23    the alternative method would be just right now.

24         Q.    Yours doesn't agree with what the Bureau did, 

25    does it?
                                                              2458

 1         A.    I beg your pardon?

 2         Q.    The yours, the variance/covariance matrix that 

 3    you calculated, doesn't agree with the Bureau's does it?

 4         A.    Let me refer to my report again, please.  

 5               May I please refer to page 33 of my report.  

 6    There is a Table 15 in the middle of the page, which I did 

 7    mention yesterday.  Below that table there is a discussion 

 8    which I believe is responsive to your question.  Freddie 

 9    Navarro, if I may refer to him in that way, and I, it says 

10    here, are using the same formula for the delta method, and I 

11    know that because I checked it with him, and we agree on the 

12    variance/covariance matrix G for state population share 

13    changes.  

14               I know that because we spent an interesting hour 

15    on the telephone reading each other numbers from his matrix 

16    and my matrix.  So I am using pretty much exactly the same 

17    variance/covariance matrix that they are.  

18               THE COURT:  That is your idea of an interesting 

19    hour?  

20               THE WITNESS:  Your Honor, statisticians have very 

21    odd ideas sometimes.

22         A.    I don't want to say that the numbers were 

23    identically the same.  But to about six significant figures, 

24    they were.

25         Q.    You can calculate these data by use of a Monte 
                                                              2459

 1    Carlo method, and you can calculate these data by use of an 

 2    analytic method.  You talked about both of these things 

 3    yesterday, correct?

 4         A.    No, I don't think that is right, at least in the 

 5    way I am using the language here.  If you want me to do 

 6    something else, please tell me.  

 7               The matrix G is an analytical object which he 

 8    computed and which I computed analytically.  He went on to 

 9    do a Monte Carlo and actually so did I.  That Monte Carlo is 

10    discussed at the bottom of page 33 and the top of page 34. 

11               There is some difference between the Monte Carlo 

12    that I did and the Monte Carlo which he did.  That is 

13    reported on page 34 in that picture and in the prose.  His 

14    number came out substantially below the number that I got.  

15    The difference has no practical significance whatsoever.  It 

16    would be of interest only to a pair of statisticians gabbing 

17    on the telephone.  

18               But his number is, I think, substantially below 

19    what you could reasonably get by a Monte Carlo, and I think 

20    he agreed with me on that point.  But we agreed totally, 

21    insofar as you can agree on these things when you exchange 

22    numbers between computers, we agreed on the analytical 

23    matrix G.

24         Q.    Both of you knew what each of you was talking 

25    about when you were talking about the analytical matrix, 
                                                              2460

 1    right?

 2         A.    I am rather sure that that is the case, or I 

 3    would not testify that way.  We talked to each other on the 

 4    telephone.  We exchanged faxes.  I may be wrong.  Anybody 

 5    can make a mistake, starting with me, but I am rather --

 6         Q.    Just so -- 

 7               MR. SITCOV:  Your Honor.  

 8               THE COURT:  He is finishing.  

 9               Go ahead.  You are finishing your answer.  That 

10    is going to be his objection.  

11               MR. SITCOV:  Yes.

12         A.    I am rather confident that we were talking about 

13    the same matrix, yes.

14         Q.    Is it fair to say that the bulk of time that you 

15    have spent studying the census and issues of adjustment has 

16    been as a retained expert for the government?

17         A.    I think that's fair.

18         Q.    You would agree that sampling survey is not a 

19    subspecialty of yours?  You would agree with that, wouldn't 

20    you?

21         A.    No, I don't think so.  I have done work on sample 

22    surveys.

23         Q.    Do you consider yourself a specialist in sample 

24    surveys?

25         A.    No, I don't consider myself a specialist in 
                                                              2461

 1    sample surveys.  But I have certainly done work in sample 

 2    surveys.  I have published papers about them.  I have done 

 3    consulting on it.

 4         Q.    With respect to the papers that you published, 

 5    you mentioned yesterday that you published two papers, only 

 6    two papers, on the issue of census or issues of adjustment.  

 7    Only one of those was in a scholarly journal, isn't that 

 8    right?

 9         A.    I think that is quite wrong.  I published one 

10    paper in Statistical Science, which is a good journal in my 

11    field.  I published one paper in Science, which is one of 

12    the great journals in North America if not the world.  I 

13    have a third paper in press in a journal called Methodology 

14    which is a house journal of Statistics Canada, which is the 

15    census bureau in my country of birth.

16         Q.    Do you consider science one of the learned 

17    journals of the statistics profession?

18         A.    I consider it one of the great learned journals 

19    which covers all fields of science, including mine.

20         Q.    Is it accurate to state that when you looked at 

21    the smoothing model with respect to the litigation in 1984, 

22    that that was your first real encounter with the smoothing 

23    model in an applied context?

24         A.    That is fair.

25         Q.    You, in fact, never structured or performed or 
                                                              2462

 1    analyzed a loss function analysis in order to make any 

 2    real-life decision, have you? 

 3         A.    That is true, and I think that would be true not 

 4    only of me but of many others as well.

 5         Q.    As a general matter, you don't publish on 

 6    population estimates or projections, is that a fair 

 7    statement?

 8         A.    That is true.

 9         Q.    Generally, in your work, you don't usually use 

10    population projections, is that right?

11         A.    I use them sometimes.  I think I testified at my 

12    deposition, and I will certainly testify today, that I have 

13    used population projections.  But they certainly don't do 

14    not form a major basis for most of the work I do.

15         Q.    Do you agree, Professor Freedman, that with 

16    respect to your writing about the uses to be made of 

17    statistics, that you have tended to take a hard-line or 

18    extremist position?

19         A.    I will accept the former as a characterization, 

20    but I will certainly not accept the latter.  When people 

21    come to me with a model, my question is, and it is my 

22    question in this case -- I am not trying to ask you a 

23    question.  I know you can't do that.

24         Q.    All I want is a answer to my question.  You have 

25    answered it.  
                                                              2463

 1               MR. SITCOV:  No.  I think the witness was in the 

 2    midst of answering it.  

 3               MR. SOLOMON:  He already answered it.  He said he 

 4    could accept with respect to hard-line, he wouldn't accept 

 5    with respect to extremist.  

 6               THE COURT:  Let's move on.

 7         Q.    It is also fair to say that you are described by 

 8    others in the field of statistics as being one who takes 

 9    hard-line or polar positions, is that a fair statement?  

10               MR. SITCOV:  I object.  

11               THE COURT:  Overruled.

12         A.    I rather think that some of the people in your 

13    side of the case would describe me that way, and I think 

14    many other people in the profession would not describe me 

15    that way.

16         Q.    Let me ask you about Professor Meier.  He is on 

17    your side of the case ostensibly, correct?  

18               MR. SITCOV:  I object.  

19               MR. SOLOMON:  He has testified on the behalf of 

20    defendants.  

21               THE COURT:  That is a better way to put it.  

22               MR. SOLOMON:  You're right.

23         A.    I think he is on our side of the case both 

24    ostensibly and for real.

25         Q.    The defendants offered, and it was admitted into 
                                                              2464

 1    evidence, an exhibit, an article by Professor Meier, DX-80, 

 2    in which he describes your position under the heading of 

 3    absolutionist or strict constructionist.  Do you think that 

 4    is a fair label?  

 5               THE COURT:  Absolutionist or absolutist?  

 6               MR. SOLOMON:  Absolutist and strict 

 7    constructionist, right.  

 8               THE COURT:  Only I give absolution here.

 9         A.    I guess generally I should say I want to know the 

10    context, I want to look at the article, I want to see that.  

11    But I think I am not going to do that.  I am just going to 

12    say no, I don't think Paul was right about that.  Whatever 

13    it was, he said it, and I wish he hadn't.  

14               But if that is his opinion, which he would have 

15    found out by asking questions in court, I wasn't hear for 

16    that, I haven't read his transcript, if that is his opinion, 

17    that is his opinion.  What can I do?  He and I are still 

18    good friends.

19         Q.    In another article offered by Professor Meier, 

20    DX-81, Professor Meier says, "I do not quite share David 

21    Freedman's hard-line defense against foremost statistical 

22    inference for observational data."  Is that a fair 

23    characterization?  

24               MR. SITCOV:  Of what?

25         A.    If that is Paul's position, that is Paul's 
                                                              2465

 1    position.

 2         Q.    Do you agree with it?  

 3               MR. SITCOV:  Mr. Solomon asked if it was a fair 

 4    characterization.  I am wondering, is he asking is it a fair 

 5    characterization of what Dr. Meier said or is it a fair 

 6    characterization of Dr. Freedman's position?  

 7               THE COURT:  Would you clarify that?

 8         A.    If Mr. Solomon is quoting from Paul Meier's 

 9    article and Paul Meier said it, then I am sure it is a fair 

10    statement of what Paul was thinking at the time.  Whether he 

11    thinks that in general, I do not know.  

12               I think I testified earlier that I take a rather 

13    hard-line position on models.  My question about a model is, 

14    why is it right?  If that is what Paul is saying, well, yes, 

15    that is my position.  I am not doing anything special for 

16    the model in this case.

17         Q.    In one of the other articles that were offered by 

18    the defendants, DX-92, there was a comment to your paper, 

19    the one paper on the 1984 litigation that you wrote, by 

20    Professor Madansky.  Do you remember that comment?

21         A.    I sure do.

22         Q.    Do you remember that he likens you to the Neturei 

23    Karta in Jerusalem?

24         A.    A great crack, yes.

25         Q.    You actually taught in Jerusalem.  You know who 
                                                              2466

 1    the Neturei Karta are, don't you?  

 2         A.    I should, but I don't think I do.  I just 

 3    remember the crack.

 4         Q.    They describe them as the ultra-extremists who 

 5    live mainly in Jerusalem, whose quest, if successful, will 

 6    mean that the state of Israel will cease to exist.  Is there 

 7    some fair analogy between you and the Neturei Karta?

 8         A.    One of your experts testified in my presence that 

 9    if my kind of position was adopted, then a lot of 

10    statisticians would have out of work.  My response to Al's 

11    comment in that article was, yes, there is a lot of bad 

12    statistics out there, and there is also a lot of good 

13    statistics, including some done by Al Medanski.  He has a 

14    famous paper on how to find the best pastrami sandwich in 

15    New York.  Bill and I did mention him.

16         Q.    When you were retained by the government in this 

17    case in 1989, that was in connection with the plaintiffs' 

18    motion for a preliminary injunction, isn't that right?

19         A.    I believe that's right, yes.

20         Q.    Since that motion was resolved, you have been 

21    retained more or less on an ongoing basis by the government, 

22    correct?

23         A.    That is true.

24         Q.    Even after the motion was resolved, you continued 

25    to advise the government, correct?
                                                              2467

 1         A.    Right.

 2         Q.    Up to the time of your deposition, I think you 

 3    told us you had earned, and I just did the calculations -- 

 4    the last one I am going to do, in fact -- about $150,000 

 5    from this retention?

 6         A.    That's right, yes.

 7         Q.    How much time have you spent since then?

 8         A.    I haven't added it up, but at a rough estimate 

 9    maybe 200 hours.

10         Q.    When you were talking with Mr. Sitcov yesterday, 

11    you said that you visited the Bureau in 1987.  At the time 

12    it was really before you had formed much of an opinion on 

13    the adjustment question.  Do you recall testifying to that?

14         A.    Yes let me be clear.  I had a very clear opinion 

15    about '84 or '80, but I had not really formed an opinion 

16    about '90.

17         Q.    So prior to that time you had, in fact, published 

18    your article about the smoothing in the 1984 litigation, 

19    correct?

20         A.    Yes.

21         Q.    It was also in 1987 that you testified before 

22    Congress, correct?

23         A.    Right.

24         Q.    At that time you said, among other things, for 

25    one thing, people aren't fish, in trying to differentiate 
                                                              2468

 1    the problems you thought there would be in the dual-system 

 2    estimation in the 1990 census.  Do you recall that?

 3         A.    I do, and I would even be willing to testify to 

 4    that today.

 5         Q.    I don't think we have to disagree on that.

 6         A.    That is another point of contact between me --

 7         Q.    Also in 1987 you expressed concern about the 

 8    matching that might go on in the 1990 census, didn't you?

 9         A.    No.  I expressed concern about the matching that 

10    would go on in the post-enumeration survey following the 

11    census.

12         Q.    Following the 1990 census?

13         A.    Following the 1990 census, right.

14         Q.    You also complained about the smoothing 

15    techniques that were then available in 1987 -- 

16               MR. SITCOV:  Objection.

17         A.    I won't accept the term "complained."

18         Q.    You also criticized the smoothing techniques that 

19    were being talked about for the 1990 census, is that fair?

20         A.    Yes.

21         Q.    Before Congress you, in fact, made a 

22    recommendation in 1987 that we should decide now not to 

23    adjust, is that right?

24         A.    That is correct.

25         Q.    Was that right around the time that you hadn't 
                                                              2469

 1    formed your opinion?

 2         A.    Oh, no.  That was after the time I had spent at 

 3    the Bureau and after the time I had received the documents, 

 4    had read them, had had some exchange of views with Kirk 

 5    Wolter.  In fact, when I testified in front of Congress that 

 6    time, the committee wanted those letters, and Kirk and I 

 7    agreed to turn them over.

 8         Q.    It was you who suggested Professor Meier as a 

 9    witness for the defendants in this case, isn't that correct?

10         A.    I believe that's right.

11         Q.    It was you who suggested Professor Breiman as a 

12    witness for the defendants in this case, correct?

13         A.    I believe that's right.

14         Q.    It was you who suggested Professor Wachter as a 

15    candidate for the Special Advisory Panel, correct?

16         A.    Again, I think that's right.

17         Q.    In fact, I saw you sitting at counsel table for a 

18    good part of the first couple of weeks of the trial, didn't I?

19         A.    Your eyes do not deceive you.  I was there.  

20               MR. SOLOMON:  With apologies, I have to talk 

21    about smoothing.

22         Q.    In connection with your own work, you don't 

23    perform smoothing analysis, correct?

24         A.    I think that's right, but I would say that with 

25    respect to the 19484 case or the case about the 1980 census 
                                                              2470

 1    and the case with respect to this case, I have certainly 

 2    done a lot of smoothing and presmoothing as well.

 3         Q.    It is your opinion that smoothing is a 

 4    controversial issue because its operation depends on 

 5    statistical models whose assumptions have not been validated 

 6    in your opinion, isn't that right?

 7         A.    I am not sure that the operation depends on it, 

 8    but certainly the conclusions you draw from the operation of 

 9    the Bureau would depend on the assumptions, and that is, 

10    indeed, a reason why I think it is controversial.

11         Q.    Are you aware that the Bureau used smoothing in 

12    connection with the generation of revenue sharing estimates?   

13    Are you aware of that?

14         A.    I became aware of it in the 1984 litigation when 

15    one of the counsel representing plaintiffs told me about it 

16    on the stand.  I am perfectly happy to accept that they do, 

17    if they do.

18         Q.    According to you, I take it that the assumptions 

19    underlying that use of smoothing also around validated, is 

20    that right?

21         A.    I can't being make any blanket statements about 

22    models that I really haven't looked at.  It could easily pe 

23    the case that the assumptions are not validated, and I would 

24    find it problematic if I looked at it.  But I certainly do 

25    not want to testify about that particular model.  I think 
                                                              2471

 1    that is probably the Fay-Herriot model, and I do not have an 

 2    informed expert opinion about it.  I have opinions about 

 3    classes of models perhaps, but I can't really testify about 

 4    that particular model.

 5         Q.    In your opinion, have you ever met a smoothing 

 6    model that you liked?  

 7               THE COURT:  Like Will Rogers?

 8         A.    I met two, and I sure don't like them.

 9         Q.    You are not aware of any that you do believe  are 

10    validated, correct?

11         A.    Well, I'm not sure that is quite right.  I heard 

12    John Rolph's testimony about his fire alarm caper.  As best 

13    you can tell about a model with a five-minute introduction 

14    in a courtroom setting, especially from a witness on the 

15    other side, I liked what I heard, because he checked it 

16    against reality.

17         Q.    Thank you.  We don't know the true undercount 

18    estimate.  We don't know the true undercount, correct?

19         A.    We certainly do not.

20         Q.    So if what you were requiring was testing and 

21    validation in the way you described what Dr. Rolph talked 

22    about, then we would never have validation of the Bureau's 

23    smoothing techniques, is that a fair statement?

24         A.    I'm not sure it's totally fair.  You can try to 

25    validate these models by using them in situations which seem 
                                                              2472

 1    quite analogous to the situation that you are primarily 

 2    interested in.  So I think it would be hard to validate 

 3    these smoothing models in any strict way.  That is certainly 

 4    part of the reason why I am skeptical about them.  In that 

 5    respect, I think you are fairly characterizing my position.  

 6               But I have seen rather little work in any attempt 

 7    to validate these models in analogous contexts.  

 8               (Continued on next page) 

 9    

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2473

 1         Q.    But you were not --  you didn't work at the 

 2    Bureau in the 1980s, correct?

 3         A.    It seems to me I did a tiny bit of consulting for 

 4    them in the sense of going there a few times and giving them 

 5    talks and discussing issues with them.

 6               Other than that, certainly not.

 7         Q.    Although you talked yesterday with Mr. Sitcov 

 8    repeatedly about what the Bureau did and what the Bureau was 

 9    thinking, you can't really do anything other than speculate 

10    about what they were talking about and what they were 

11    analyzing within the Bureau, is that a fair statement?

12         A.    Oh, I think that's quite wrong, because I do talk 
                                                                   

13    to them on the telephone and I do visit there and I do know 

14    all those peoples and I see them at professional meetings.  

15    If I were speculating here, I would try to label it as 

16    speculation.

17               I may be wrong about this or that, but I have not 

18    testified to anything which in my opinion can fairly be 

19    described as speculation.  I testified as to what I had 

20    heard.  

21               Now, you may not like to admit that kind of 

22    testimony and I can't speak to that, that's a legal matter, 

23    but what I am testifying to yesterday and today is what I 

24    have heard from people at the Bureau and what I have read.

25         Q.    There were some articles published about the 
                                                              2474

 1    smoothing that was done as part of the tests, field tests 

 2    prior to the 1990 census.  Did you read those articles?

 3         A.    I can recall an article by Don Ylvisaker that I 

 4    read.  There were some papers on the smoothing in TARO.  I 

 5    cannot remember the authors.  I read some of those.

 6               I cannot recall very much about the St. Louis 

 7    test and I do not recall sharply whether or not I read 

 8    papers about the smoothing there.

 9         Q.    You talked yesterday about smoothing and 

10    presmoothing and then benchmarking as if they were all 

11    different and acceptable techniques.

12               You are aware that each of those was prespecified 

13    in the way we have been using that term in this litigation 

14    prior to the 1990 census, is that correct?  

15               MR. SITCOV:  I object.  I'm not sure that Dr. 

16    Freedman knows how Mr. Solomon is using the term 

17    prespecified.  

18               MR. SOLOMON:  He spent the day yesterday 

19    commenting on the evidence by all the witness.  It has been 

20    defined and used by all those witnesses.

21         Q.    If you have an understanding, you can answer it.  

22    If not, I will clarify it.  

23               THE COURT:  The objection is overruled.  You may 

24    answer.

25         A.    I have an understanding and I will answer.  I 
                                                              2475

 1    can't believe that the presmoothing and benchmarking is 

 2    prespecified and I will even tell you why.

 3         Q.    I didn't ask you why.  Thank you.

 4               I want to read you from the testimony of Dr. Fay 

 5    and ask you whether you have any basis to disagree with 

 6    that.

 7               "And the smoothing and the presmoothing 

 8    operations were almost completely prespecified, weren't 

 9    they?

10               "A.    Close to completely." 

11               The question that I have for you is whether you 

12    have any basis in fact to disagree with Dr. Fay?

13         A.    I do have a basis and I do disagree with him.  

14    I'm sorry to disagree with Bob Fay, who is a person I 

15    greatly like and respect, but I disagree with him and I have 

16    a basis for it.

17         Q.    Tell us what parts were not prespecified and then 

18    tell us the basis.

19         A.    I do not believe that the presmoothing was 

20    prespecified --  look, before you push the button, it's 

21    prespecified.  Everything is prespecified before you push 

22    the button.  

23               But the presmoothing was not prespecified any 

24    great time in advance of the publication of the June 13 

25    press release.  I have a document in evidence here that 
                                                              2476

 1    shows that in about March the Bureau was ascribing a 

 2    presmoothing model which was not the model that they used.  

 3    In fact, there is testimony about it in court.  I think we 

 4    pointed to a logarithmic equation that this document was 

 5    referring to as presmoothing and that is not the equation 

 6    that was ultimately used.

 7               The benchmarking was not prespecified.  There 

 8    were some extensive rangle about that, to my understanding, 

 9    in the Bureau and with special advisory panel and I base 

10    that on two facts:

11               First of all, Ken Wachter told me about it and 

12    second of all they were floundering around trying to compute 

13    the variances after benchmarking.  If benchmarking were 

14    prespecified they would have known how to compute the 

15    variances.

16         Q.    Did you hear the testimony here that the Bureau 

17    had decided upon a method for presmoothing seven months 

18    before the time that you are now talking about ?  Did you 

19    hear that testimony?  That's my question?

20         A.    I'm going to try to answer it.

21               I heard, I think I heard to my recollection John 

22    Rolph explain the general idea of presmoothing was 

23    prespecified.  That is the only testimony that I can recall 

24    that speaks to that issue.

25         Q.    You testified yesterday that you believed that 
                                                              2477

 1    bias is introduced by smoothing, is that correct?

 2         A.    I believe it and so do many of the other experts 

 3    who testified here.

 4         Q.    You have never attempted to verify if smoothing 

 5    introduces bias, isn't that a fair statement?

 6         A.    That is not a fair statement.  If you look at the 

 7    corrections to my deposition transcript, which we supplied, 

 8    I explained to you that in my 1986 statistical science 

 9    article, Bill Navidi and I offered mathematical prove that 

10    smoothing can introduce bias.  

11               I think I also added to the corrections --  I 

12    believe, I hope in the corrections I said that some of the 

13    computer experiments we did with variable selection 

14    algorithms points to the possibility of bias introduced by 

15    smoothing.

16               What is I think fair to say is that we have not 

17    quantified the amount of the bias in 1990.

18         Q.    Just so the judge understands, the deposition 

19    that you took and the answers that you gave are on page 303.

20               "Q.    Do you have a conclusion about how much 

21    bias is introduced by smoothing?

22               "A.    No.

23               "Q.    Did you do any studies to attempt to 

24    quantify how many biases is introduced by smoothing?

25               "A.    Not as a separate item.
                                                              2478

 1               "Q.    So you have never attempted to varify 

 2    whether, in fact, smoothing introduces bias?

 3               "A.    I don't remember having done so."  

 4               MR. SITCOV:  Your Honor, if this is used as 

 5    impeachment, which is what I think Mr. Solomon is doing, 

 6    then he should also read the correction page that Dr. 

 7    Freedman testified about just a moment ago and that was 

 8    provided to plaintiffs within the 30-day period.  

 9               MR. SOLOMON:  The long answer that the witness 

10    says he added was his correction to perfectly clear 

11    questions that were made.  I don't have any need to do more.  

12    If you want to do it on redirect, you are welcome to it.  

13               THE COURT:  You can do it on redirect, but we are 

14    here now, let's take care of it.  There may not be any 

15    redirect.  

16               MR. SITCOV:  That's right.  If Mr. Solomon has 

17    the correct --  

18               THE COURT:  Would you read me the corrected 

19    version?  

20               MR. SOLOMON:  I will be happy to.  Let me get a 

21    version.

22               I assumed it was correct when he said it.  

23               MR. SITCOV:  I move to strike it, your Honor.  

24               THE COURT:  I will strike that.  

25               MR. SOLOMON:  Bear with me one moment while I get 
                                                              2479

 1    it.  

 2               THE COURT:  All right.

 3               (Pause) 

 4               MR. SOLOMON:  I am prepared to take the witness' 

 5    recollection, he just testified at length as to what he said 

 6    he did.  I don't have any problem with that.  

 7               THE COURT:  All right, let's move on.  

 8    BY MR. SOLOMON:

 9         Q.    One of the points that you were making yesterday 

10    is that in your opinion additional sources of variance exist 

11    in the dual system estimation process other than are capture 

12    in the total error model.  Is that a fair statement?

13         A.    I'm not sure it's quite fair.

14               I think what I was saying was that, first of all, 

15    I thought the variances as measures of sampling error were 

16    substantially underestimated by the Bureau.  That was point 

17    one.

18               And that point two, there are sources of error in 

19    the dual system estimates other than sampling error which 

20    are not captured by the variances.

21               Now, some --  I'm sorry, I do want to try to 

22    finish this answer because it is a complicated thought and 

23    I'll lose it.

24               The other sources of error are not quantified at 

25    all in the variances, but many of them, in my opinion not 
                                                              2480

 1    all, perhaps even not even important ones, but many of them 

 2    are captured, I think was the term you used, in the total 

 3    error model and reflected in the loss function analysis.

 4               So my position is that the variances measure only 

 5    sampling error and they understate those.

 6               Many, but not all, and not even all important 

 7    sources of error, are captured in the total error model and 

 8    reflected in the loss function analysis.

 9         Q.    I will ask you about variances.  You answered 

10    that question very early in the questions.

11         A.    You asked me about --  

12               THE COURT:  Let's not fight with each other.

13         Q.    With respect to your estimate of the additional 

14    variance, your best estimate of the additional variance is 

15    about two times, is that a fair statement?

16         A.    As a measure of sampling error, yes.

17         Q.    And all the work that you have done and all the 

18    opinions you expressed yesterday was really based on an 

19    assumption that the variance was about two times more than 

20    what it had been estimated it to be, is that right?

21         A.    No, I don't think that's right at all.

22         Q.    To the extent you were talking about variance 

23    yesterday, have you ever calculated a variance that you can 

24    justify greater than two times?

25         A.    I think so.  For example, with that alternative 
                                                              2481

 1    assumption, if you take that, then you get variances for 

 2    shares which are nearly three times what the Bureau's 

 3    variances are.

 4         Q.    Thank you.

 5               Have you calculated anything greater than three 

 6    times?

 7         A.    I don't believe so.  Speaking strictly as a 

 8    sampling error, the answer to that is no.

 9         Q.    I would like you to turn, please, to DX 64, 

10    again, the loss function paper, to the glossary, way at the 

11    back, page 2.

12               Let me ask you just as a predicate whether I was 

13    understanding how the various papers fit together.

14               The shorter loss function paper you talked about 

15    --  Professor Freedman, are you all right?  We can go off 

16    the record.

17         A.    No, I am fine.  My eyes are bad so I can't see 

18    both you and the document.

19         Q.    Okay, fine.

20               The first paper, the short paper, talks about 

21    changing the variables that are used in the smoothing and 

22    you used the results of that in the larger smoothing paper 

23    and you used the results of that in your loss function 

24    paper.  Is that generally correct?

25         A.    Well, Exhibit 63 is about alternative variability 
                                                              2482

 1    selection procedures and some of those results are used in 

 2    Exhibit 62 and some of the calculations in Exhibit 62 feed 

 3    into Exhibit 64.  

 4               It's a little confusing because the discussion of 

 5    sampling error in Exhibit 64 is really the front end to 

 6    Exhibit 62.

 7         Q.    Okay.

 8         A.    I apologize for the confusion.

 9         Q.    Okay.  That clarifies it.

10               Now I am on page 2 of the glossary.

11         A.    Yes, sir.

12         Q.    And you have the equation there for the dual 

13    system estimator.

14         A.    Yes.

15         Q.    The EE term there, what is the EE term?

16         A.    EE is the estimated number of erroneous 

17    enumerations from the post-enumeration survey.

18         Q.    And that is a sample estimate, is that right?

19         A.    It is.

20         Q.    Are you aware that the Bureau took that EE and 

21    modified it before including it in the formula?

22         A.    I'm aware that the Bureau uses a different 

23    formula for 1990 than this formula.  This is the right 

24    formula, I think, for 1980, it is not the right formula for 

25    1990.
                                                              2483

 1               I'm not quite sure what you are referring to.

 2         Q.    This is not the right formula for 1990 is what I 

 3    am referring to, but I thank you.

 4               I would like you to talk to the small paper, the 

 5    short smoothing paper, DX 63.  I would like you to turn to 

 6    page 18, under the heading, "Smoothing the covariance 

 7    matrices." 

 8               Do you have that?

 9         A.    Yes.

10         Q.    Is this the formula that you used to smooth the 

11    covariance matrices, or as you were calling it yesterday, 

12    presmooth, in this short paper?

13         A.    In the short paper, yes, not in the long paper, 

14    but in the short paper.

15         Q.    And you know that this is an incorrect formula, 

16    that is, it is not the formula the Bureau, in fact, used, is 

17    that a fair statement?

18         A.    I do not accept at all the characterization that 

19    it is incorrect.  It was chosen as an alternative.  It is 

20    rather close to the procedure that was described in Cary 

21    Isaki's paper of March.

22               It is certainly right to say that it is different 

23    from the method the Bureau used for the production dual 

24    system estimator.

25         Q.    Now, you say that in the larger smoothing paper 
                                                              2484

 1    you changed the formula for presmoothing, is that right?

 2         A.    In the main paper, when we are replicating the 

 3    Bureau's model, we used their procedure for presmoothing.

 4               When we present results from this paper, of 

 5    course, the presmoothing is done the way it is done in this 

 6    paper.

 7         Q.    But what I want to be clear on is that in your 

 8    larger smoothing paper, when you do the presmoothing, when 

 9    you calculate the presmoothing under the Bureau method, you 

10    are not using the log equation, is that right, you are using 

11    a different equation?

12         A.    I'm using the Bureau's equation.

13         Q.    You are aware that the Bureau considered the log 

14    equation which you use in your first smoothing paper, aren't 

15    you?

16         A.    Not only am I aware of it, that's why we did it 

17    like --  well, that's not quite right.  Part of the reason 

18    we did it was because it was a suggestion of the Bureau's 

19    and partly it's because my proposal for presmoothing drove 

20    Richard Cutler crazy, he just wouldn't do it that way.

21         Q.    Are you aware that the Bureau rejected this 

22    formula for use in presmoothing?

23         A.    Obviously, because they presmoothed a different 

24    way.

25         Q.    Do you agree that the reason they rejected it was 
                                                              2485

 1    because this formula requires more ratio adjustment, more 

 2    benchmarking than the formula that was, in fact, used?  

 3               MR. SITCOV:  I object.  

 4               THE COURT:  Overruled.

 5         A.    No, I don't really know why the Bureau chose the 

 6    method that they chose over this logarithmic equation.

 7               I do agree that the benchmarking of the variances 

 8    is an argument against these methods, but I'm not --  I 

 9    don't know that that's why they made the choice that they 

10    made.

11         Q.    What you are testifying to is that you agree by 

12    using the method that you used, you have to do more 

13    benchmarking than, in fact, the Bureau's method?

14         A.    If I said that I was clumsy.  What I was trying 

15    to say is I agree that the need to benchmark the predicted 

16    variances is a flaw in these methods.

17               What I do not know and I will certainly accept 

18    your statement of the fact if you are telling me that, is 

19    that with this method there is more benchmarking than with 

20    the other method.

21         Q.    Let me show you PX 145.  

22               MR. SOLOMON:  May I, your Honor?  

23               THE COURT:  Yes.

24               (Handing to the witness)

25               (Pause)
                                                              2486

 1         Q.    I would call your attention to the second page, 

 2    capital B, under modeling the estimated sampling variance.

 3               Tell me if I am correct that the log formula 

 4    which you use is rejected here and the Bureau says of that, 

 5    "The ratio factor for the nonlog dependent variable was much 

 6    closer to the one, so we decided to use the one that they, 

 7    in fact, used as the dependent variable."

 8               Do you see that, sir?

 9         A.    Are you asking me if I see what they have written 

10    about that?

11         Q.    That's the first question, that's all I am 

12    asking.

13         A.    Yes, I do see it.

14         Q.    Does that mean that the log equation that you 

15    used for presmoothing requires more benchmarking than the 

16    one that the Bureau, in fact, used?

17         A.    No, it does not, and the reason is that the 

18    equation we are using is different in detail from the 

19    equation that they are using.

20               Their equation and our equation involved logs and 

21    they involved regression, but the detailed formulation of 

22    the equations is different.

23               Our equation, I think, is explained as you were 

24    pointing me to at the bottom of page 18 and their equation 

25    is explained in this right-hand column of type in this 
                                                              2487

 1    exhibit you handed me, and at least to a statistician there 

 2    are clear differences between the two methods.

 3               Now, I don't want to go beyond what I know.  It 

 4    could be that with my equation, too, there is more 

 5    benchmarking than with the Bureau's equation, but I do not 

 6    know that and it does not follow from what you have showed 

 7    me.

 8         Q.    You didn't check to see whether yours, the 

 9    formula that you were using required more benchmarking than 

10    the Bureau's before you used it in this paper, is that 

11    correct?

12         A.    That I must concede.

13         Q.    Did you anyplace in this paper explain that what 

14    you were using as the presmoothing was not, in fact, what 

15    the Bureau was using?

16         A.    May I just take a look, please?

17         Q.    Of course.

18               (Pause)

19         A.    May I ask you, please, to look at the top of page 

20    19, but below the equation there, and it says, "This 

21    method," in other words, the method of presmoothing that we 

22    have just discussed in the paper, "This method is similar to 

23    the one used by the Bureau, but differs in detail."

24         Q.    Is that it?

25         A.    That's it.
                                                              2488

 1         Q.    One more --

 2         A.    Let me look at the other report for a second, 

 3    please.

 4               (Pause)

 5         A.    No, I think that's all.

 6         Q.    You agree, don't you, Professor Freedman, that in 

 7    fitting the smoothing model, the choice of variables is 

 8    critical?

 9         A.    Critical, I think, is too strong, but I do think 

10    that it matters.

11         Q.    It matters a lot, is that a fair statement?

12         A.    I think that is fair.

13         Q.    And you are aware that the Bureau choose the 

14    variables for the smoothing that was done as part of the 

15    PES, correct?

16         A.    Yes, sir.

17         Q.    And the method that was chosen was the best 

18    subsets regression, correct?

19         A.    Well, not quite.  You have to throw in Colin 

20    Mallows, and the method is called the capital C sub lower 

21    case P method.

22         Q.    Is it your testimony that it is unfair to 

23    characterize the Bureau's choice of variable method as the 

24    best subsets regressions method?

25         A.    I think it is.  I think you have to throw in this 
                                                              2489

 1    Mallows C sub P business.

 2         Q.    Okay.

 3               And the variables that you use in the paper and 

 4    is the basis on which you testified to yesterday, those 

 5    variables are different from the variables the Bureau used, 

 6    correct?

 7         A.    Well, I don't think that's fair.

 8               What we did is we started with the exact same 

 9    variables that they started with, in fact, they supplied 

10    them to us, and then we used a different variable selection 

11    rule.

12               So what is quite fair to say is that our final 

13    equation has different variables in it from their equation.

14         Q.    Okay.

15               You say that you started with exactly the same 

16    variables, but you used a different choice rule.

17               What rule did you use?

18         A.    Well, I thought it was explained in the paper, 

19    but if I can try to summarize it here, what we did was --  

20    well, sorry, let me back up to the beginning.

21         Q.    Why don't I just rephrase the question.

22               I withdraw the question.  

23               THE COURT:  Yes.

24         Q.    I think I can just speed it along.  If you think 

25    you described it yesterday, that's why I am raising it now.
                                                              2490

 1               You look for variables with T ratios greater than 

 2    2, is that correct?

 3         A.    Yes.

 4         Q.    And in what you did, you included all of the 

 5    Bureau's variables with T ratios greater than 2, is that 

 6    correct?

 7         A.    Well, that's the part that's not quite right and 

 8    I was trying to explain that.

 9               That is, they have some variables which are built 

10    up out of other variables by multiplication, subtraction and 

11    so on and so forth, and so we discarded all those composed 

12    variables.  That left us with, so to speak, their initial 

13    stock.

14               Then we applied this variables selection rule 

15    that you described, namely, that if the T statistic was 

16    bigger than 2 we kept it.

17               I think there was a case on refitting there were 

18    insignificant variables and then we made a second pass.

19               So that is our algorithm.

20         Q.    So in creating your variables, you took only some 

21    of the Bureau's variables that had T ratios greater than 2, 

22    correct?

23         A.    We took the variables that had a T ratio bigger 

24    than 2 on the second pass, so that would exclude some that 

25    had a T ratio bigger than 2 on the first pass, that's right.
                                                              2491

 1         Q.    And do you know that you excluded by doing that, 

 2    I don't know, between three and six in each of the various 

 3    regions that you calculated variables for?  Is that a fair 

 4    estimate?

 5         A.    I'm really not with you.

 6         Q.    You said that your method of selection was to 

 7    look for variables with T ratios greater than 2, but you 

 8    didn't take all the variables with T ratios greater than 2, 

 9    even some variables with T ratios greater than 2 you 

10    discarded?

11         A.    No, no, we fitted the equation again and if in 

12    the second pass the T statistic was smaller than 2, we 

13    omitted them.

14               Now, I can't remember in how many regions that 

15    happened.  I believe it happened in the northeast and we 

16    eliminated two variables on that basis.

17               If you want me to look at the paper, we do report 

18    all that and I can count them for you.

19         Q.    You talked about the cumulative variables, right?  

20    The Bureau used those, right?

21         A.    Cumulative?

22         Q.    It is just easier to show you the document.

23               I would like to show you DX 69.

24               Do you have this in your binder, DX 69?

25         A.    Yes.
                                                              2492

 1         Q.    One example will proof the point.

 2               Will you turn, please, to page 6.

 3               This document, by the way, sets out the variables 

 4    that the Bureau used, correct?

 5         A.    Yes.

 6         Q.    And on page 6 are the variables that the Bureau 

 7    used for the south region, correct?

 8         A.    Yes.

 9         Q.    And when you did your analysis, you discarded 

10    F 25, F 107, F 108, F 109, F 113, isn't that right?

11         A.    Well, I'm sure we discarded, without looking at 

12    anything, I'm sure we discarded F 107, because that's one of 

13    those composed variables which is obtained by multiplying 

14    the indicator for Houston, Dallas and other large central 

15    cities with percent of Hispanic, so that's what I called a 

16    composed variables and we for sure eliminated that.

17               We may well have eliminated variables such that 

18    the Bureau's T statistic exceeded 2.

19               We were not using their T statistics, we were 

20    using our own T statistics, and if I was unclear about that, 

21    I'm glad to have a chance to clarify it.

22         Q.    Okay.  So you discarded the ones that I mentioned 

23    and you also performed your own T statistics and then 

24    discarded some more, is that right?

25         A.    No, no, no.  We did not look at their T 
                                                              2493

 1    statistics in choosing our variables.  What we did is we 

 2    started with their list of variables, we discarded what I am 

 3    calling the composed or derived variables, of which a good 

 4    example was F 107 in the south or F 108 or F 109 or F 113.  

 5    We then ran a generalized least squares regression using a 

 6    variance covariance matrix somewhat different from the 

 7    Bureau's.

 8               If in that regression the T statistic was less 

 9    than 2, we discarded the corresponding variable and we 

10    refitted, and in a couple of cases where the T statistics 

11    were less than 2 in the second pass we discarded those.

12         Q.    Do you agree, Professor Freedman, that the reason 

13    the Bureau sought to smooth and presmooth and benchmark was 

14    to reduce the effects of sampling error while preserving to 

15    the maximum extent possible the meaningful systematic 

16    differences among the adjustment factors?

17         A.    Well, that's somebody's version of the events.  I 

18    am not sure I want to adopt that as my own testimony.

19               I believe they smoothed to reduce sampling error.  

20    The presmoothing and the benchmarking I think is about 

21    something else.  

22               THE COURT:  Shall we take a little recess here?  

23               MR. SOLOMON:  Yes.  

24               THE COURT:  We will resume at 11.

25               (Recess) 
                                                              2494

 1               THE COURT:  Mr. Solomon.  

 2               MR. SOLOMON:  Thank you, sir.  

 3    BY MR. SOLOMON:

 4         Q.    Professor Freedman, you told us there was one 

 5    formula for presmoothing using one of the papers and another 

 6    in another of the papers.

 7               I would like to ask you a couple more questions 

 8    about the presmoothing formula that are being used here that 

 9    you testified to yesterday.

10               In terms of the choice of variables for the 

11    modeling of the variances for the presmoothing part of this 

12    process, the Bureau used what?

13         A.    Beats me.  As far as I can tell, they just picked 

14    those variables.

15               I asked Cary Isaki how he did it and he said, "I 

16    just picked them."

17         Q.    And what did you do in your papers?

18         A.    Well, let's take them separately.

19               In Exhibit 63, which is the alternative models 

20    for smoothing adjustment factors, the rule we used was to 

21    use exactly the same variables for the presmoothing that we 

22    were using in the regression model for smoothing, so we only 

23    had one variables selection algorithm.

24         Q.    Okay.  And in the larger paper?

25         A.    In Exhibit 62, we used exactly the variables that 
                                                              2495

 1    the Bureau used.

 2         Q.    I see.  So you knew which variables the Bureau 

 3    used?

 4         A.    Let me try and find an exhibit if I may.

 5               (Pause)

 6               Yes, they are listed in Exhibit 63.

 7         Q.    So you now know which the Bureau used.

 8               Did you use the same variables for the 

 9    presmoothing as the Bureau did?

10         A.    Well, I have to differentiate between the two 

11    papers.

12               In the alternative selection rule paper with Dan 

13    Costa and Richard Cutler, we followed the rule that I said, 

14    namely, for presmoothing we used exactly the same variables 

15    that we used for smoothing.

16               In the smoothing paper itself, when I was trying 

17    to replicate the Bureau's model and in the simulations and 

18    so forth, I used the variables that the Bureau used for 

19    presmoothing.  

20               And to complete the answer let me just refer to 

21    that list of files that we were looking at.

22               (Pause)

23               The handwritten notes in that bundle of papers 

24    that is PX 766 gives the layout to the files, and the file 

25    names, although the Xeroxing is not so good, are V SMOG
                                                              2496

 1    X 1.DAT and so forth.  V SMOTH X 1.DAT, that's for region 

 2    one, and there are similar files for two, three, four and 

 3    five.

 4         Q.    I wasn't asking you about that, but thank you.

 5               Let's talk about outliers.

 6               By the way, you referred to California yesterday 

 7    as an outlier.  You surely did not mean that in any 

 8    statistical sense, correct?

 9         A.    I don't recall --  well, California is an outlier 

10    in many respects.

11               I don't recall referring to it as an outlier.  I 

12    recall saying that I thought California's estimated risk 

13    numbers were so big that they skewed the loss function 

14    analysis, but I don't recall referring to it as an outlier.

15         Q.    What outlier method did the Bureau use?  This is 

16    with respect to the modeling of the variances, the 

17    presmoothing part of the process.

18         A.    I'm not sure they used a method.  They selected 

19    some of the ones, I think they are the big ones in each 

20    region, and those are listed in one of these exhibits, 69.

21         Q.    Did you use the same method in your papers?

22         A.    Well, I have to answer separately for the two 

23    papers.

24               In the paper on alternative selection models, no, 

25    we did not, we used our own method, and since we took logs, 
                                                              2497

 1    I don't think we had big outliers, maybe we did, I don't 

 2    remember that.

 3               In the main paper on smoothing, we followed in 

 4    most of the analysis exactly the same principles that the 

 5    Bureau did, we used exactly their formulas, as best I could 

 6    determine it.

 7               In the simulations we did not use a rule for --  

 8    no, that's going to be wrong.

 9               In the simulations we did have outliers in the 

10    variance world that we had to deal with and we dealt with 

11    them by a different principle.

12               Our problem was not big variances, but due to the 

13    way we were simulating them we came up with some negative 

14    variances and we truncated those below at a convenient small 

15    positive number.

16         Q.    By a different method in your last answer, you 

17    just mean different from the way the Bureau did it, correct?

18         A.    In the simulations, yes.

19         Q.    Thank you.

20               With respect to ratio adjustment of the formula 

21    for the variances, modeling the variances in the 

22    presmoothing, okay, what did the Bureau do with that? 

23               The Bureau did ratio adjust, didn't they?

24         A.    I believe so.

25         Q.    Okay.
                                                              2498

 1               What did you do?

 2         A.    In the main part of the paper where we are 

 3    replicating their model, we did exactly the same thing and 

 4    we checked that we were able to replicate their presmoothed 

 5    variances.

 6               In the simulations, we ratio adjusted the way the 

 7    Bureau did, that is, we ratio adjusted the simulated 

 8    predicted presmoothed variances to have the same sum as the 

 9    simulated real variances.

10               I don't know if that's such have a clear answer.  

11    I can try again if you like.

12         Q.    You did exactly what the Bureau did in your 

13    simulations?

14         A.    As far as the ratio adjustment, yes.

15               As far as the outliers, no.

16         Q.    All right.

17               With respect to the factors themselves, the 

18    adjustment factors themselves, the Bureau had a rule for 

19    treating outliers, correct?

20         A.    It did.

21         Q.    And you did not follow that rule in your 

22    simulations, correct?

23         A.    That is right.

24         Q.    And in terms of the method of estimation, the 

25    estimation method, the Bureau used a maximum likelihood 
                                                              2499

 1    method for the adjustment factors, correct?

 2         A.    Yes.  The Bureau used maximum likelihood.

 3         Q.    And you used a different one, called method of 

 4    moments, correct?

 5         A.    That is certainly true for the main tables in the 

 6    paper.  

 7               We did some checks to see whether there was any 

 8    difference between method of moments and maximum likelihood.  

 9    That is also reported in the paper.  

10               In the simulations we ran we saw little 

11    difference, except in one case.  In that case I was not sure 

12    the code was right for other reasons.

13         Q.    And with respect to ratio adjustment of the 

14    adjustment factors or benchmarking, the Bureau did ratio 

15    adjust and you didn't, correct?

16         A.    In the --  on the paper where we are discussing 

17    their results, replicating their results, sometimes I 

18    benchmarked, sometimes I didn't, and the notes to the tables 

19    try to explain which is which.

20         Q.    Okay.

21         A.    In the simulations we did not --  I did not 

22    benchmark and I think that is indicated, too, in the notes 

23    to the table.

24         Q.    With respect to whether you are going to use 

25    sigma squared I or your preference V sub 3, I would like to 
                                                              2500

 1    ask you a couple of questions about that.

 2               The one that you have used, V sub 3, has the 

 3    effect of making the variance covariance matrix of the true 

 4    exactly proportional to the variance covariance matrix of 

 5    the raw adjustment factors, correct?

 6         A.    That's right, although I don't so much like 

 7    characterizing it as my preference.

 8         Q.    Okay.  The one you talked about yesterday?

 9         A.    It sure is.

10         Q.    All right.

11               And the effect of the one that you talked about 

12    yesterday pulls adjustment factors the same distance to the 

13    regression model, all of the adjustment factors, correct?

14         A.    Well, that would be true if you were using it to 

15    adjust the population.  However, that is not what I was 

16    using that matrix for.

17               What I was using that matrix for was as an 

18    alternative basis for computing the standard errors of the 

19    Bureau's procedures.  In other words, I am following the 

20    Bureau in adjusting the population of the United States as 

21    they did, but I am running a sensitivity test on their 

22    method for computing the standard errors.  

23               So that alternative, the scissors and scotch tape 

24    alternative is offered not as a method for adjusting the 

25    United States, but as an alternative way of computing the 
                                                              2501

 1    standard errors.

 2               If we were to adjust the United States that way, 

 3    I think what you said is quite right, that it would be 

 4    shrunk down by a constant factor.

 5         Q.    Okay.

 6               Are you aware that the Bureau considered and 

 7    rejected the V sub 3 or a substantively equivalent 

 8    adjustment factor?  Yes or no?

 9         A.    I've been told that.

10         Q.    And I take it that it is correct that you never 

11    talked with Dr. Fay about your proposed adjustment factor 

12    prior to the time that the Bureau used sigma squared I, is 

13    that correct?

14         A.    I think that's right, yes.

15         Q.    You also know that Dr. Fay prefers what the 

16    Bureau did to what you did?  You know that, too, don't you?

17         A.    No, I don't, but if you tell it to me, I will 

18    accept what you say.

19         Q.    I will ask you --  it's transcript page 1938.

20               "In your view, Dr. Fay, is that a preferable 

21    alternative to the assumption used by the Bureau in the 1990 

22    smoothing?

23               "A.    No, it's not preferable."  

24               MR. SITCOV:  I object.  I'm not certain what the 

25    purpose is of reading to --  
                                                              2502

 1               MR. SOLOMON:  I haven't asked him a question.  

 2               MR. SITCOV:  He is simply reading to the witness 

 3    Dr. Fay's testimony and the witness --  

 4               THE COURT:  I presume he is going to ask him to 

 5    comment on it.

 6               Is that basically it?  

 7               MR. SOLOMON:  Yes.  

 8               THE COURT:  Comment briefly.  

 9    BY MR. SOLOMON:

10         Q.    No.  You don't have any basis to disagree with 

11    that, do you?

12         A.    Well, if that's what Bob Fay things, that's what 

13    Bob Fay things and I was happy to take your word for it, but 

14    it's not what I think.  I mean, it isn't -- .

15         Q.    You disagree with the Bureau.  All right.

16         A.    Oh, I disagree with the Bureau in many 

17    particulars, including that one.

18         Q.    Let's talk about loss functions if we can for a 

19    few minutes.

20               You agree that a loss function is a conventional 

21    summary statistic?

22         A.    I think that's right.

23         Q.    All right.  And in your opinion, it would be 

24    unreasonable to --  if you are doing a loss function 

25    analysis, it would be unreasonable to ignore areas that have 
                                                              2503

 1    a lot to gain and a lot to loss merely because they have a 

 2    lot to gain and a lot to loss?  Do you agree with that 

 3    statement?

 4         A.    I'm not sure the amount of gain or loss in terms 

 5    of population share or dollars or Congressional 

 6    representation plays a large role in my thinking.  The 

 7    amount of gain or loss in accuracy plays some role.

 8         Q.    But you agree that if what we are talking about 

 9    is gain or loss not in the sense of accuracy but as a result 

10    of a correction, you agree that it is illegitimate to 

11    disregard large gains or large losses merely because they 

12    are large gains or large losses, don't you?

13         A.    I'm not sure we are agreeing with each other 

14    here.

15         Q.    Okay.

16         A.    That is, I think that it is good to pay attention 

17    to gains in accuracy, large or small, and thing about them 

18    and thing about whether the loss function is a good summary 

19    statistic.  

20               I'm not sure that it is such a good idea to think 

21    about the actual gains or losses in terms of dollars and 

22    Congress and so forth.  

23               I'm not sure it is a bad idea to think about 

24    that, either.

25         Q.    Am I correct that in your opinion, if one could 
                                                              2504

 1    show that adjustment was going to move the shares in the 

 2    right direction by a substantial amount and only for a few 

 3    states would it be worse and that the amounts by which they 

 4    would be made worse is small, that in your opinion, you 

 5    would favor adjustment?

 6         A.    That's right.  I was adopting Fisher's testimony, 

 7    Dr. Fisher's testimony. 

 8               If the moves in the right direction are a lot 

 9    more numerous and a lot bigger than the moves in the wrong 

10    direction, then that's a good basis for adjustment.

11         Q.    Even if a few states were made a little bit 

12    worse, that wouldn't stand in the way of adjustment, in your 

13    view, right?

14         A.    That is correct.

15         Q.    I would like to show you an exhibit.  

16               MR. SOLOMON:  May I, your Honor?  

17               THE COURT:  Yes.

18               (Handing to the witness)

19               (Pause)

20         Q.    That for identification is PX 757.

21               You recognize this, don't you, Professor 

22    Freedman?

23         A.    I surely do.

24         Q.    And what this shows is a hypothetical adjustment 

25    technique, the original enumeration counts, in essence, a 
                                                              2505

 1    total of five for each of five areas, for the total of five 

 2    areas, if the adjustment moves us to nine, but the truth is 

 3    at ten, correct?

 4         A.    Right.  That's for total.

 5         Q.    Total.

 6               And in this example, states or units B, C and D 

 7    are made worse off, disimproved by the proposed adjustment, 

 8    correct?

 9         A.    Well, B, C and D are --  sorry, may I just do the 

10    arithmetic again?  I know I saw this exhibit before, but I 

11    would like to do the arithmetic.

12         Q.    Go right ahead.

13               (Pause)

14         A.    Okay, now I have done the arithmetic, but I have 

15    lost the question.

16         Q.    No problem.

17               This shows that as a result of the adjustment, 

18    states B, C and D or units B, C and D are disimproved, 

19    correct?

20         A.    States B, C and D have their --  well, I don't 

21    have such a simple answer to that, I'm afraid.  

22               This is an interesting example, because states B, 

23    C and D have the accuracy of their shares made slightly 

24    worse or somewhat worse, however, they actually gain from 

25    adjustment.  That's one interesting feature of this example 
                                                              2506

 1    which was presented to me earlier.

 2         Q.    Right.  You say they gain because in terms of 

 3    strict numeric accuracy they are better numbers by the 

 4    adjustment, correct?

 5         A.    No, actually, I was looking at shares.

 6         Q.    Okay.

 7         A.    And I observe that the shares for B, C and D are 

 8    made slightly worse in terms of accuracy, that is, the 

 9    census or original enumeration gave them each 20 percent, 

10    the truth is 20 percent, the adjustment moves them up to 22 

11    percent, so that introduces a little or, I don't know if I 

12    should say little, that introduces an inaccuracy, but 

13    interestingly, it is an inaccuracy which actually works to 

14    the benefit of B, C and D because they gain population 

15    share.

16         Q.    What I am saying is, just so the court 

17    understands it, as a result of this hypothetical, states B, 

18    C and D which were each at 20 percent before the adjustment, 

19    and we know the truth is 20 percent for each, each of those 

20    move away from 20 percent and, therefore, are made more so, 

21    is that correct, in terms of the accuracy of the shares?

22         A.    Right.  In terms of the accuracy of the shares, 

23    states B, C and D are made worse.

24         Q.    And state E is very substantially improved, 

25    though it doesn't go all the way to truth, correct?
                                                              2507

 1         A.    Yes, states A and E are improved in terms of the 

 2    accuracy of their shares, B, C and D are made worse.

 3         Q.    And in this hypothetical, you would agree to 

 4    adjust, correct?

 5         A.    Yes, I would.  The improvement for A and E is 

 6    substantial, the disimprovement, if I can adopt that word 

 7    for the moment, for B, C and D seems relatively small.  

 8               B, C and D seem to suffer no practical harm, 

 9    indeed, some practical gain from adjustment and, 

10    furthermore, and this is a feature which I think is really 

11    important and is not captured in the Bureau's loss function 

12    analysis, the population ratios, that is, A to B, B to C, 

13    A to D and so forth, are made better.

14               The example is a little unrealistic, of course, 

15    because, well, there are several features, including --

16         Q.    I am not asking you that, sir.  You can be asked 

17    that on redirect.

18               In your opinion, if someone looking just at this 

19    hypothetical were to thing that one shouldn't adjust, you 

20    feel comfortable saying he or she was borderline 

21    unreasonable, though you are not going to tell us which side 

22    of the border, is that a fair statement?

23         A.    That's fair.

24         Q.    Okay.

25               I would like to have you look at page 6, figure 1 
                                                              2508

 1    of your loss function paper.

 2         A.    Do you mean figure 1 on page 6?

 3         Q.    Yes.  Page 6, figure 1.

 4               Do you have it?

 5         A.    Yes.

 6         Q.    Okay.

 7               What this shows is that according to the Bureau, 

 8    there are some states for which we are highly confident that 

 9    an adjustment would improve them and a large number of 

10    states where we cannot tell, is that correct?

11         A.    I'm not sure that's quite right.  It's close.

12               What we see here is there are a lot of states 

13    where you can't tell whether the proposed adjustment is just 

14    reflecting sampling error or is reflecting real differential 

15    overcounts and undercounts.

16               There are a number of states, however, where 

17    there is clear statistical evidence that the proposed 

18    adjustment is not just reflective of sampling error.  

19    Whether adjustment would be improving the shares is yet 

20    another question.

21         Q.    Okay.

22               It would be incorrect to state that with respect 

23    to the lines in the middle that cross zero, improvement will 

24    make the shares worse, isn't that right?

25         A.    I think you mean adjustment would make the shares 
                                                              2509

 1    worse rather than improvement.

 2         Q.    I do, yes.

 3               That would be wrong to say, correct?

 4         A.    Well, we don't know for sure.  I think for 

 5    confidence intervals that cross zero, I would have very 

 6    little personal belief in the efficacy of adjustment, 

 7    because we haven't even demonstrated that the census is 

 8    wrong.

 9         Q.    So you don't have any basis to believe that they 

10    are, in fact, made worse, you are uncertain, is that right?

11         A.    Well, we are to some extent uncertain even about 

12    states where the confidence interval fails to cross zero, 

13    but we are, we are uncertain about all of these states.  The 

14    uncertainty is if the improvement from adjustment seems to 

15    me to be very large for those states where the confidence 

16    interval crosses zero.

17         Q.    If on the basis of this data we didn't adjust, I 

18    take it what we are saying is that we have no more 

19    confidence about California and Texas and New Mexico and 

20    Arizona and then the ones on the bottom than we do about all 

21    the other ones, is that a fair statement?

22         A.    No, I did not mean to say that, I don't think I 

23    did say that.

24               I think what this chart shows is that for a large 

25    majority of the states we don't even have any convincing 
                                                              2510

 1    evidence on the table that the census is wrong, never mind 

 2    that adjustment will make it better.  

 3               For some substantial states but still a minority, 

 4    taking the theory of adjustment at its face value, there is 

 5    quite reasonable evidence that the census was wrong.

 6         Q.    But was it on the basis of these numbers that the 

 7    Bureau calculated its loss function analyses?

 8         A.    I don't think that is quite right.  What is true, 

 9    I think, is that some of the raw material for figure 1 on 

10    page 6 feeds into the loss function analysis.  However, you 

11    need additional information, namely, information about the 

12    biases in the dual system estimator which is not reflected 

13    on this page at all.

14         Q.    And there is one other difference, and that is 

15    that you have changed the standard errors in these numbers 

16    from what the Bureau calculated, is that correct?

17         A.    Look, the Bureau in its press release of June 13 

18    got the formula for the standard errors wrong.  When they 

19    went and did the loss function analysis, I believe they got 

20    the formula right.

21         Q.    All of the Bureau's loss function analyses 

22    favored adjustment, isn't that correct?

23         A.    On their face I think that's right.

24         Q.    Okay.

25               Lit me ask you a couple of questions.
                                                              2511

 1         A.    If I may, I would like to amend a previous 

 2    answer.

 3               It's not quite right to say that the Bureau made 

 4    a mistake in its press release of June 13, they were in a 

 5    big rush to get it out as far as I understand and they 

 6    deliberately took a shortcut.  They knew what they were 

 7    doing, they knew they were taking a shortcut.

 8         Q.    I would like to show you what Mr. Sitcov 

 9    yesterday showed you as PX 686.

10               Do you have that up there?

11         A.    I have it.  

12               MR. SOLOMON:  Your Honor, do you have it?  

13               THE COURT:  No.  

14               MR. SOLOMON:  You can have this one.

15               (Handing to the court)

16               (Pause) 

17    BY MR. SOLOMON:

18         Q.    You recall talking to Mr. Sitcov about this 

19    yesterday?

20         A.    I recall testifying about it.

21         Q.    Okay.  And this is the chart that Dr. Fisher 

22    talked about, correct?

23         A.    Right.

24         Q.    And what you said was in trying to explain in a 

25    rough and ready way was that if you assumed that the census 
                                                              2512

 1    was perfect, which you took just as a hypothetical, 

 2    according to this chart, an adjustment would be making all 

 3    of the states' shares --  and you assume --  let me start 

 4    again.

 5               If you assume that the census was perfect so that 

 6    adjustment by definition is going to make all the states' 

 7    shares worse, you testified that a procedure that was 

 8    reflected in this chart would suggest that approximately 

 9    half of the states would be expected to improve.

10               Do you recall that generally?

11         A.    I do.

12         Q.    And you said that the reason why you thought what 

13    you believed to be an error was made was that the people 

14    calculating this ignored the variability in the dual system 

15    estimator; correct?

16         A.    That's right.

17         Q.    Okay.

18               Now, you testified that you heard or you read 

19    Professor Fisher's testimony.

20               Do you recall that he said that he based this 

21    chart on a memorandum by the Bureau, the Spencer to Mulry 

22    memorandum?  Do you recall that?

23         A.    Yes.

24         Q.    Did you review that memorandum?

25         A.    No.
                                                              2513

 1         Q.    You just reached a conclusion that they must have 

 2    ignored variability in the dual system estimator without 

 3    reading the memorandum that Professor Fisher said he relied 

 4    on, is that correct?

 5         A.    That's correct.  I heard his testimony and I 

 6    relied on his testimony.

 7         Q.    Okay.

 8               I am going to show you the memorandum.           

 9               MR. SOLOMON:  May I?  

10               THE COURT:  Yes.

11               (Handing to the witness)

12               (Pause)

13         Q.    Let me ask you to look at page B 7166.

14         A.    Yes.

15         Q.    And you see in the second to the last equation on 

16    the page, it's the absolute value minus U is less than DS, 

17    subscript minus T is less than the absolute value of U?  Do 

18    you see that?

19         A.    Yes.

20         Q.    I want you to assume as Professor Fisher said in 

21    his testimony this is the memorandum that he was relying on, 

22    you can tell, can't you, that if the census was perfect as 

23    you suggested in your hypothetical, that that equation would 

24    not improve any state, all you would have would be zeros?

25         A.    I'm having a little trouble following the 
                                                              2514

 1    notation.

 2               I think what you are saying is right.

 3         Q.    Okay.  What the absolute value is less than 

 4    something is, again, less than the positive absolute value, 

 5    so it's the same absolute value, so if the census is 

 6    perfect, then the two hands clapping are zero and nothing 

 7    fits in between, right?  

 8               THE COURT:  You did say the two hands clapping?  

 9               MR. SOLOMON:  The two hands, the two Us on both 

10    sides of this inequality.  This is inequality A is less than 

11    B is less than C and A and C are the minus whatever the 

12    undercount is and plus whatever the undercount is and the 

13    purpose is to see whether the estimate fits in between, and 

14    as Professor Freedman said yesterday if you assume the 

15    census is perfect, then, of course, there is zero undercount 

16    and zero less than something is less than zero and you never 

17    get anything.

18               I just want to confirm that that is basically 

19    right.

20         A.    Well, I'm not ready to commit, but I think it's 

21    right.

22         Q.    Okay.

23               Look at the next formula underneath that, in 
      particular, the reference to V as in victory.
24               Do you see that?
           A.    I do.
25               (Continued on the next page) 
                                                              2515

 1         Q.    That notation that is being used here is a 

 2    reflection of the variability in the dual-system estimator, 

 3    isn't it?

 4         A.    Come on.  The fact that it is in the formula 

 5    doesn't mean that you are properly taking it into account.

 6         Q.    Did you want to answer my question?  Then you can 
                                                                   

 7    ask yourself questions on redirect.  

 8               It is in the formula, isn't it?

 9         A.    Yes.  The standard deviation of the DSE is in the 

10    formula. 

11         Q.    So if, as Professor Fisher said, he relied on 

12    this, then it is incorrect to state that he didn't take the 

13    variability of the dual-system estimator into account, is 

14    that fair?

15         A.    I don't think that's right.

16         Q.    Let me ask you next to look at PX-698, which was 

17    the other chart that Mr. Sitcov talked to you about.  Do you 

18    have that?  With respect to this one, I believe you 

19    testified to Mr. Sitcov that -- 

20               MR. SITCOV:  Your Honor, I think I am going to 

21    have to move to have some of this stricken.  Mr. Solomon has 

22    been representing that Dr. Fisher testified about 

23    Plaintiffs' Exhibit 711.  It is our understanding that 711 

24    has never been before the Court before.  

25               MR. SOLOMON:  711?  
                                                              2516

 1               MR. SITCOV:  Yes, 711.  

 2               MR. SOLOMON:  I don't know what you are talking 

 3    about.  What is 711?  

 4               MR. SITCOV:  711 is what you just handed us.  

 5               MR. SOLOMON:  No, no.  What I said, your Honor, 

 6    was, I asked this witness whether he had heard the 

 7    testimony.  The witness was asked, what was the basis on 

 8    which he created this chart, PX-686.  Professor Fisher at 

 9    transcript 1125, "It was a memorandum, Bruce Spencer to Mary 

10    Mulry, generated sometime around the beginning of this year 

11    or the end of last year, describes how that was done."  

12               All I asked the witness to do was to assume that 

13    that was the memorandum.  I haven't offered it.  

14               MR. SITCOV:  I did not hear him to assume that 

15    was the memorandum.  I heard him to assume some things about 

16    the memorandum.  I certainly think that that sort of 

17    questioning is quite unfair of this witness.  Are they 

18    offering this as the memorandum that Dr. Fisher relied on or 

19    not?  I think that the witness is entitled to know that, as 

20    are we.  

21               MR. SOLOMON:  I am entitled on cross-examination, 

22    I believe, your Honor, to ask the witness hypotheticals.  I 

23    can ask him to assume this.  In the transcript Professor 

24    Fisher says it's a memorandum, Spencer to Mulry, dated 

25    sometime around the beginning of this year or the end of 
                                                              2517

 1    last year.  And who knows, we may have a rebuttal case, and 

 2    who knows, Professor Fisher may in fact be able to connect 

 3    it up.  Even if he can't, I am entitled to ask.  

 4               THE COURT:  Dr. Freedman is blithely doing his 

 5    equations there, having fun.  

 6               Are you aware that we have no proof that PX-711 

 7    is, in fact, the document that supplied the data that led to 

 8    686?  So your last set of questions was based on the 

 9    assumption that 711 contained the data.  Are you aware of 

10    that?  

11               THE WITNESS:  I am now.  

12               THE COURT:  Do you want to change anything you 

13    said based on that?  

14               THE WITNESS:  I do not.  

15    BY MR. SOLOMON:

16         Q.    Will you look at PX-698.  

17               THE COURT:  "Rejected"?  

18               MR. SOLOMON:  Right.

19         Q.    As to this, you told Mr. Sitcov that you believe 

20    that Professor Fisher made a similar mistake in that he 

21    didn't estimate the variability of the dual-system 

22    estimator; is that basically what you said?

23         A.    No.  I think what I said, and I think I'm right, 

24    and I will say it again today, what I think he did -- he 

25    didn't do it.  Whoever was computing these numbers for him, 
                                                              2518

 1    I think, failed to take into account the variability in the 

 2    dual-system estimator but, rather, took into account the 

 3    variability in the targets.

 4         Q.    You testified yesterday that you, on the basis of 

 5    two slightly different methods, succeeded in following his 

 6    recipe in getting the number that he came out with, correct?

 7         A.    I did.

 8         Q.    Tell us as precisely as you can recall what those 

 9    slightly different methods were.

10         A.    Sure.  I listened to his description of the 

11    method, the critical ingredient being that you get some list 

12    of numbers which he described and which I suppose I can 

13    reconstruct, and you take the standard deviation of that 

14    list.  I asked Bob Fay to do that on the data files that are 

15    the basis for this chart.  He did, and he came up with a 

16    number of something like 5.65, if I remember properly.  It 

17    was immaterially different from 5.68.  

18               I also asked Bob to provide for me certain 

19    summary data on these files, more specifically the 

20    variance/covariance matrix for the targets, for the thousand 

21    targets that are the basis for this calculation.  

22               Then I computed by a theoretical method what the 

23    results of computing the standard deviation on the list of a 

24    thousand numbers was.  I came up with the number which I 

25    think was just under 560, in part, I think, because I am 
                                                              2519

 1    using a slightly different version of the data than Dr. 

 2    Fisher is.  Or it may just be that when you transfer these 

 3    numbers from one machine to another they lose a little 

 4    accuracy.

 5         Q.    Did you produce to us the results of what Dr. Fay 

 6    did?

 7         A.    The only result I know of is the calculation of 

 8    the standard deviation.  I am not sure I remember what it 

 9    is.

10         Q.    Did you give that to us in the disk that you gave 

11    to us?

12         A.    No.

13         Q.    You then said that you did your own calculation.  

14    Did you give that to us?

15         A.    I don't think so, no.  

16               MR. SOLOMON:  Your Honor, we would like both of 

17    those produced under 705, asit formed the basis of an 

18    opinion that he offered yesterday.  We cannot check it.  I 

19    have not been hounding you much today on that subject.  

20    Much.  

21               THE COURT:  It is never off my mind.  

22               Mr. Sitcov, can you accommodate him?  

23               MR. SITCOV:  Well, that is certainly just 

24    rebuttal testimony that he offered to Dr. Fisher.  I think 

25    we can accommodate him in any event.  
                                                              2520

 1               THE COURT:  Thank you.

 2         Q.    On this chart that you are looking at, PX-698, 

 3    the source that is given for it is the final loss function 

 4    analysis.  Do you see that, sir?

 5         A.    I do.

 6         Q.    I ask you to assume that that is correct.  Is it 

 7    correct that the final loss function analysis did estimate 

 8    the variability in the dual-system estimator?

 9         A.    No, no.  I think it is fairer to say that the 

10    final loss function analysis took into account estimated 

11    variability in the dual-system estimator.

12         Q.    That is fair to say?

13         A.    Oh, I think so, yes.

14         Q.    That is what you believe based on the checking 

15    that you have done that Dr. Fisher did not do, is that 

16    correct?

17         A.    No.  It seems to me he is talking about here the 

18    variability in the estimated risk difference.  The estimated 

19    risk difference has two major sources of variability in it, 

20    one from variability in the dual-system estimator and one 

21    from variability in the targets.  

22               He took the variability of the targets into 

23    account but not the variability in the dual-system 

24    estimator.  And he picked much the smaller of the two 

25    sources of variance.  That is my testimony.
                                                              2521

 1         Q.    Right.  But the Bureau, in the final loss 

 2    function analysis, did it correctly, right?

 3         A.    That is a much closer call.  My testimony 

 4    yesterday was that I thought they had made a small and not 

 5    material error in their calculation of the loss from 

 6    adjustment.  

 7               A much more serious question is whether they got 

 8    the variance/covariance matrix for the smoothed adjustment 

 9    factors right, and that is a question of a factor of 2 

10    perhaps.  And there is a further question as to whether they 

11    got those state level biases right.  Those I think are quite 

12    serious questions.

13         Q.    Thank you.  But all I was asking was in the final 

14    loss function analysis, the Bureau didn't make the mistake 

15    that you believe Dr. Fisher made, correct?

16         A.    Right.  That's fair.

17         Q.    If you are right, the source on this page is 

18    wrong?

19         A.    No, not at all.  That is not my testimony at all.  

20    My testimony is whoever made this chart up made a mistake in 

21    using those numbers.  They used somehow the wrong idea to 

22    get started on the calculation.  

23               The variance you want is in there, but the person 

24    who did this didn't get at it for the purpose of computing 

25    the standard deviation of the estimated risk difference.  
                                                              2522

 1    That is what I am trying to say.

 2         Q.    Thank you very much.  

 3               I want to turn to page 35 of your loss function 

 4    report.  You testified yesterday --

 5         A.    Okay.  I am on page 35.

 6         Q.    You testified yesterday that you performed tests 

 7    of significance in lieu of a loss function analysis, 

 8    correct?

 9         A.    I don't know in lieu of.  I think in addition to.

10         Q.    All right.  You talked with Mr. Sitcov about page 

11    39 of this document, correct?

12         A.    Yes.

13         Q.    And the formula for what you have calculated on 

14    page 39 appears on page 35, correct?

15         A.    In part.  There is a chi-square statistic there.  

16    Then you have to do some minimization, which is discussed on 

17    page 36 in order to get the numbers which are on page 39.

18         Q.    The formula to calculate the chi-squared 

19    statistic is on page 35, correct?

20         A.    Right.

21         Q.    I want to spend a minute to see whether we can 

22    understand what you have done here.  The X in the formula in 

23    number 1, formula number 1, that is the dual-system 

24    estimator of the undercount, correct?

25         A.    Well, no, not exactly.  What it is is it is the 
                                                              2523

 1    dual-system estimator for the share or the change in shares 

 2    for the states.

 3         Q.    The U with a little line down the left, the mu, 

 4    that is the assumed real undercount, correct?

 5         A.    No.  It is the assumed real share or share 

 6    change.  If you want me to sort of sharply distinguish 

 7    between shares and share changes, I will have to read the 

 8    paper and try to remember.

 9         Q.    No.  In any event, it is the assumed real share 

10    of the change in the undercount and the X is the estimate of 

11    the share change in the undercount, correct?

12         A.    X is the estimate and mu is the truth.  It is 

13    either a share or a share change.

14         Q.    What you are telling here on page 35 is in the 

15    first instance just assume that mu is equal to zero, that 

16    is, that the census got all 51 shares correct, right?

17         A.    Right.

18         Q.    And on the basis of that you calculate a 

19    chi-square statistic of 338, right?

20         A.    Right.

21         Q.    You then go on and you say that you are going to 

22    change the formula.  You say the census is better than 

23    adjustment for area I when, and you then have a formula.  

24    You then have a fortiori when, and you have another formula, 

25    right?
                                                              2524

 1         A.    Right.

 2         Q.    The second formula says, in words, and correct me 

 3    if I am wrong -- you will -- the absolute value of mu is 

 4    less than the absolute value of X over 2, is that right?

 5         A.    Right.

 6         Q.    That, you think, always follows from equation 

 7    number 1, the absolute value of mu is less than the absolute 

 8    value of X minus mu, right?

 9         A.    No.  Perhaps my Latin is terrible, but I thought 

10    it was the other way around.  That is, I thought that when 

11    the absolute value of mu was less than the absolute value of 

12    X over 2, the first equation followed, and then the census 

13    was better.

14         Q.    So the second equation does not always hold; you 

15    agree with that, correct?

16         A.    Oh, sure.

17         Q.    Do you use the second equation in calculating the 

18    chi-square statistics that you report four pages later?

19         A.    I don't know if that is quite fair.  I used the 

20    logic of that sentence.  Then to test the hypotheses to see 

21    whether these things are holding or not holding, I make the 

22    substitution discussed in the next line, where it says put 

23    mu equals X over 2 in 1.

24         Q.    When you put mu to equal X over 2, mu, we said 

25    before, was the truth, right?  That is a real nonvariable 
                                                              2525

 1    number, correct?

 2         A.    Right.

 3         Q.    X over 2 is a variable number, right?

 4         A.    That is true.

 5         Q.    What you are asking here is whether one variable 

 6    number is greater than another variable number over 2, is 

 7    that right?

 8         A.    No.  What we are asking for is whether there is a 

 9    certain relationship holding between the parameters in the 

10    problem, which is the mus, and the X in the problem, which 

11    is the random data.  And I think I even tried to write that 

12    out algebraically on page 40.  That is why technically 

13    speaking this is a simultaneous confidence interval rather 

14    than strictly speaking a hypothesis test.

15         Q.    I would like to take an example where mu is equal 

16    to X over 2.  

17               MR. SOLOMON:  Sorry.

18         Q.    See if we can work through the formula.  We can 

19    do it quickly.  Let me get something to write on.  

20               I don't suppose you are going to offer to do 

21    this, are you, Professor?

22         A.    Sure.  Just tell me what you want done, and I 

23    will do it.

24         Q.    That would be great.  I would like to work 

25    through one example of your mu equals X over 2.  Just so the 
                                                              2526

 1    judge is clear, when you put mu equals X over 2, that is 

 2    what you are telling the Court to do in order to test for a 

 3    51-state analysis whether the adjusted is better than the 

 4    unadjusted, it is your test of significance, correct?

 5         A.    No, I don't think that's right.  It is what I am 

 6    suggesting we do plus some additional minimization and 

 7    maximizeation, I forget how I set it up, to get the chi- 

 8    square tests listed on page 39.

 9         Q.    Okay.

10         A.    Incidentally, I do think that X and mu are share 

11    changes rather than shares.  I believe it says that on page 

12    29.

13         Q.    What you are telling us is that what you came up 

14    with on page 39 is derived from putting mu to equal X over 

15    2?

16         A.    True.

17         Q.    I would like to go through an example of putting 

18    mu to equal X over 2.  

19               Equation number 1 can be written -- which is X 

20    minus mu to the slash G to the minus 1, that is the same as 

21    putting G underneath in the denominator, right?

22         A.    Sort of.  These are matrices, not numbers.  So 

23    you can't really put it in a denominator.  But it is sort of 

24    right.

25         Q.    Can I put it in the denominator or not?
                                                              2527

 1         A.    No.  

 2               THE COURT:  So you are going to do it anyway.  

 3               MR. SOLOMON:  I want to know whether it is right.

 4         Q.    That is what we are doing.  We are setting mu to 

 5    X-2.  Where as in your first example you set mu to X-zero.  

 6    That is all we have done so far.

 7         A.    Right.

 8         Q.    We haven't done anything so far.  

 9               I want to replace each of the places are mu shows 

10    up, I want to replace that with X over 2, right?  I just 

11    replace the mu with X over 2.  Is that what I should do?

12         A.    Hang on a second.  I misspoke earlier.  For page 

13    39 you replace many of the mus about X over 2 and then do 

14    some minimization over the rest.  

15               If what you are talking about is the test that is 

16    described on page 35, that is, where improvement occurs for 

17    all 51 areas, then e yes, you put mu for X over 2 for all 51 

18    areas.

19         Q.    You are saying that this is used, at least in 

20    part, in deriving what you did on page 39, which you showed 

21    the judge yesterday, right?

22         A.    Yes.  It is the second line in the table.

23         Q.    Just for a one-state example, if I can figure out 

24    what this location should be, every time I see the mu, I am 

25    going to replace it with X over 2.  So I go mu equals X 
                                                              2528

 1    minus X over 2, X minus X over 2 over G, for a one-state 

 2    example.  Is it accurate when I have put mu to equal X over 

 3    2 to rewrite your equation that way?

 4         A.    Yes.

 5         Q.    This is really just saying X minus a half of X, 

 6    right?

 7         A.    Right.

 8         Q.    That is a half X?

 9         A.    Right.  

10               THE COURT:  It's not the only one.

11         Q.    This is also fairly rewritten half X?

12         A.    So far we are in sync.  I don't know what is 

13    going to happen next, but so far it is fine.

14         Q.    You don't have a problem rewriting that as X over 

15    2 squared over G?  I haven't done anything.  All I have done 

16    is grouped these together and squared them?

17         A.    Right.

18         Q.    Because the two of these I have multiplied 

19    together and they are both the staple.

20         A.    Right.

21         Q.    I would fail your statistics course, wouldn't I?

22         A.    No.  You are doing great.

23         Q.    We can rewrite this too, without, I think, 

24    changing anything at all.  Correct me if I am wrong, as X 

25    squared over 4 over G?
                                                              2529

 1         A.    Yes, sure.

 2         Q.    That is equal to X squared over 4 G?

 3         A.    Right.

 4         Q.    So far?

 5         A.    Sure.

 6         Q.    I haven't done anything that is not in your 

 7    formula, correct?

 8         A.    Right.  That is why you are dividing by 4.  Do 

 9    you see the 338 over 4?  That is where the 4 comes from.

10         Q.    Absolute right.  When you go from 338 -- you say 

11    338 divided by 4, and that is 85?

12         A.    Correct.

13         Q.    What is G?

14         A.    G is the variance/covariance matrix for the 

15    estimated change in shares from the production deal system 

16    estimator.

17         Q.    G is the variance/covariance -- does anybody use 

18    that notation?  I just want to use that for variance/ 

19    covariance.

20         A.    It's fine for the moment.

21         Q.    That is the variance/covariance matrix of X, 

22    correct?

23         A.    Right.

24         Q.    What is the variance/covariance matrix of X over 

25    2?
                                                              2530

 1         A.    It is one-quarter G.

 2         Q.    So the variance/covariance matrix of X over 2 is 

 3    one-quarter G.  The reason for that is when I cut X in half, 

 4    the variance is reduced by a full quarter, correct?

 5         A.    Right.

 6         Q.    In your equation here, when I change mu to X over 

 7    2, you used the variance/covariance matrix of X, not the 

 8    variance/covariance matrix of X over 2, correct?

 9         A.    That's right, and I am not inclined to change.

10         Q.    If we were to take the variance/covariance matrix 

11    of X over 2, which is what you told me to put into the top 

12    formula, instead of the variance/covariance matrix of X, 

13    then what we got over here would be X squared over 4 G over 

14    4, correct?

15         A.    Well, it seems to me you have now made sort of 

16    two substitutions of a factor of 2.  If you want to do that, 

17    yes, you are going to get X squared over --

18         Q.    I want you to agree with me.  If I use the 

19    variance/covariance matrix of X over 2 rather than the 

20    variance/covariance matrix of X, then this is a proper way 

21    to write the equation?

22         A.    That is exactly what I am not prepared to give 

23    you.  It seems to me that the procedure I am following here 

24    is the standard procedure for getting multiple comparison 

25    intervals of this kind.  In fact, there is a little 
                                                              2531

 1    algebraic demonstration on page 40 of the narrative of what 

 2    I am doing.  What you are doing, I think, is you are 

 3    dividing by the 2 twice, if I am following you.

 4         Q.    Agree with me this far, that what I am doing is 

 5    replacing the variance/covariance matrix for X with what you 

 6    have told me is the variance/covariance matrix for half X.

 7         A.    I will certainly give you that the 

 8    variance/covariance matrix for X over 2 is G over 4.  If you 

 9    want to substitute X over 2 for mu in the equation and G 

10    over 4 for G in the equation, if you want to do that, you 

11    will get G over 16, yes.

12         Q.    This?

13         A.    X prime G X over 16.

14         Q.    I don't presume to correct you.  I have written 

15    this correctly, right?  The G was four times too large if 

16    you are talking about the G for X over 2 rather than the G 

17    for X, right?  So I need to divide this G by 4?

18         A.    All I am prepared to give you at the moment is 

19    that if you want the variance/covariance matrix for X over 

20    2, it is G over 4.

21         Q.    All right.

22         A.    I am not yet prepared to go further than that.

23         Q.    All right.  So far so good.  If we use G over 4 

24    rather than G, then it is fair to take this equation, this 

25    number, which we came up with before, X squared over 4 G and 
                                                              2532

 1    just divide it by 4, right?

 2         A.    I am prepared to make another concession, which 

 3    is that if you take my chi-square statistic and you, A, 

 4    replace mu by X over 2.

 5         Q.    Which is what you do.

 6         A.    Let me finish the concession, then you ask 

 7    another question.

 8         Q.    Okay.

 9         A.    What I am trying to concede is that if you will 

10    replace mu by X over 2 in the chi-square statistic and then 

11    replace G by the variance/covariance matrix for X over 2, 

12    you will get a division by 16 instead of a division by 4.  I 

13    am even giving it to you in a multidimensional case.  I am 

14    not going an inch further than what I just conceded yet.

15         Q.    I think it may be enough.  For my purposes, it is 

16    accurate simply to rewrite this by crossing these two things 

17    out, isn't it?

18         A.    Maybe I am making a terrible mistake in my 

19    concessions.  I thought you would get X squared over 16 G.

20         Q.    Can we just agree that it is simply 

21    mathematically, without attributing anything else to this, I 

22    can rewrite this triple decker fraction as X squared over G?

23         A.    Actually, you are right and I'm wrong.  Let me 

24    modify my concession.  I'm terrible at doing algebra, 

25    especially on the stand.  
                                                              2533

 1               If you replace the X by the X over 2, you will 

 2    get a factor of a quarter.  Then if you replace the G by the 

 3    variance/covariance matrix for X over 2, G goes down by a 

 4    factor of 4, so G goes up by a factor of 4, and so it 

 5    cancels.  You saw that and I didn't, and I apologize.

 6         Q.    That's right all right.  We are now in agreement 

 7    that this X squared over G is the way that we would write it 

 8    if we replaced mu with X over 2 and correspondingly we took 

 9    the G not for X but the G for X over 2?  Do you agree with 

10    that?

11         A.    I agree that if you made those two modifications 

12    in the formula for chi-squared -- I am just following like a 

13    little mechanical person here -- if you make those two 

14    changes in the formula, you will get the answer that you 

15    have written down.

16         Q.    If we make the replacement of the G rather than 

17    using it for X, using it for X over 2, using the G that 

18    corresponds to X over 2, which is a quarter G you told us -- 

19    if we do that, then am I correct that the chi-square 

20    statistics that you calculate and use on page 39 are low by 

21    a factor of 4?

22         A.    What I am prepared to concede is that if you make 

23    the changes that you seek to make in the formula, you will 

24    multiply the chi-squares -- it may not be exactly 4, but 

25    close enough.
                                                              2534

 1         Q.    The only change that I am making that you and I 

 2    may have a dispute about is that I am using the variance/ 

 3    covariance matrix associated with X over 2.  We don't have 

 4    any other dispute, correct?

 5         A.    It seems to me we have a major dispute about how 

 6    to compute confidence intervals.  Now that you have 

 7    corrected my mistake in arithmetic here, we do not have a 

 8    dispute about arithmetic.  We have a dispute about how to 

 9    compute confidence intervals.  

10         Q.    You were mentioning before that if you agreed 

11    with the substitutions that I suggested.  I am only talking 

12    about one change, not two.  That is what I want to be clear 

13    on with you.  You made the first substitution because you 

14    wanted mu to equal X over 2, correct?

15         A.    No, I don't want mu to be equal to X over 2 

16    particularly.  What I want to do is test the hypotheses 

17    which are listed on page 39.  In my opinion, the appropriate 

18    way to do that is to stick with the original G and put mu 

19    equals X over 2 for some or many areas.  

20               The demonstration of that, not that it needs much 

21    demonstration, I think, is on page 4.

22         Q.    If I am correct that by replacing X with X over 2 

23    you should have replaced the G with the variance/covariance 

24    matrix associated with X over 2 rather than X, I ask you 

25    simply to assume that, then you will agree with me that the 
                                                              2535

 1    chi-square statistics on page 39 are all too low by a factor 

 2    of 4?

 3         A.    I am not sure that 4 is the right number.  But 

 4    certainly it is close to 4.  So let me try and say it again.  

 5               If I got the method for calculating confidence 

 6    intervals wrong, then all my chi-square statistics are 

 7    wrong.  I just don't happen to think that.

 8         Q.    If they are wrong in the way that I say, then 

 9    they are all too low by a factor of 4 or close to 4, 

10    correct?

11         A.    That I will gladly concede.

12         Q.    If the chi-square statistics on page 39 which you 

13    went through with the judge yesterday are all too low by a 

14    factor of 4 or roughly 4, then on the basis of what you tell 

15    us on page 38, am I correct that each of the conclusions 

16    that you draw is reversed?

17         A.    Absolutely.  If you should multiply all those 

18    chi-square statistics by 4, which is a hypothetical you 

19    require me to adopt, then, sure, all this evidence flips 

20    over.

21         Q.    It is your testimony that you believe it is 

22    appropriate to use the variance/covariance matrix for X even 

23    though what you are testing is X over 2, correct?

24         A.    No.  I think that is the fundamental problem.  

25    What I am testing are the hypotheses listed on the lefthand 
                                                              2536

 1    side of the chart Table 17 on page 39.  Substituting mu 

 2    equals X over 2 for some or many statesties only a step on 

 3    the road to making that test.

 4         Q.    In the test that we are talking about, even 

 5    though you have taken X, which is the estimate of the 

 6    undercount, and you have cut it in half, you still believe 

 7    it is appropriate to take the variance/covariance matrix for 

 8    the full X, not the X over 2; am I understanding you 

 9    correctly?

10         A.    It is my opinion -- and of course everybody can 

11    make mistakes, starting with me, but it is my opinion, 

12    despite this cross-examination, that I have got the method 

13    for computing those confidence intervals right.  So I am 

14    insisting that you should keep the original G and it is 

15    appropriate to substitute mu equals X over 2 in that formula 

16    for the purpose of testing the hypotheses in the lefthand 

17    column of Table 17.  That is my testimony.

18         Q.    Thank you.  Let me ask you one final series.  It 

19    goes something like this.  

20               If X is an estimate of an undercount and I have a 

21    variance estimate for that X of A, would you agree that if I 

22    instead changed the X to X over 2, then mathematically you 

23    can tell me that the variance for X over 2, rather than 

24    being A, is going to be A over 4?

25         A.    Sure.  I gave you that several questions ago, and 
                                                              2537

 1    I will give it to you again.  That is quite right.  

 2               MR. SOLOMON:  I have nothing further.  

 3               THE COURT:  Thank you.  

 4               MR. SOLOMON:  I neglected to mark this.  We would 

 5    offer PX-757, which is the one-page chart with the shares 

 6    that three states disimprove.  We would offer it for 

 7    illustrative purposes only.  

 8               THE COURT:  No problem?  757 is admitted for 

 9    illustrative purposes only.  

10               (Plaintiffs' Exhibit 757 for identification was 

11    received in evidence) 

12               THE COURT:  Any redirect?  

13               MR. SITCOV:  Could we take a short break?  

14               THE COURT:  Thank you.  Let's do it.  Five 

15    minutes.  

16               (Recess) 

17               THE COURT:  Mr. Sitcov?  

18               MR. SITCOV:  Thank you, your Honor.  

19    REDIRECT EXAMINATION 
                           

20    BY MR. SITCOV:

21         Q.    Do you see Exhibit 686 in front of you there?

22         A.    Yes.

23         Q.    Do you recall that Mr. Solomon asked you a couple 

24    of questions about this chart?

25         A.    Yes.
                                                              2538

 1         Q.    You testified that you think Dr. Fisher's mistake 

 2    was not taking the variability in the dual-system estimate 

 3    into account.  Do you remember that?

 4         A.    Yes.

 5         Q.    If the variance from the dual-system estimate is 

 6    in the formula, how can you say that Dr. Fisher didn't take 

 7    variability in the due system estimate into account?

 8         A.    The formula that Mr. Solomon showed me is in 

 9    Plaintiffs' Exhibit 711 on page 7266.  At the bottom of the 

10    page, I can't quite read the subscripts, but there is an F 

11    paren UV and V.  E is, if I am following the notation, the 

12    standard deviation of the dual-system estimator.  That is in 

13    the formula.  But now we do not know U.  U is the bias in 

14    the census estimate.  So knowing U is equivalent to knowing 

15    truth.  This is exactly what we do not know.  

16               My understanding of Dr. Fisher's testimony is 

17    that he simply substituted for U an estimate for U based on 

18    the dual-system estimate, and he did not take into account 

19    the variability in U and the imprecision in his estimate 

20    based on the fact that the dual-system estimate for U is 

21    only an estimate.  

22               In fact, this paper talks about a bootstrap 

23    adjustment to take care of that problem, although in my 

24    opinion the bootstrap adjustment will be probably 

25    unsuccessful.  I have written on this topic, and the 
                                                              2539

 1    bootstrap is not a good way to adjust these kinds of biases.

 2         Q.    Do you have Plaintiffs' Exhibit 757 in front of 

 3    you?  It was the one that Mr. Solomon handed you, Dr. 

 4    Freedman.  

 5               THE COURT:  757?  

 6               MR. SITCOV:  757.  It is a single page.  

 7               THE COURT:  The one you did the arithmetic on.

 8         A.    Got it.

 9         Q.    In your opinion, would you favor adjustment on 

10    the basis of the at that time a in that table?  

11               MR. SOLOMON:  Objection.  Asked and answered.  

12               THE COURT:  Overruled.

13         A.    Yes, I would.

14         Q.    Why?

15         A.    We are making two states better, A and E, by 

16    quite a margin.  We are making B, C, and D a little bit 

17    worse.  However, the states that we are making worse in 

18    terms of accuracy, namely, B, C, and D, we are giving them a 

19    little more population, which is presumably in their 

20    interest.  

21               Finally, a feature that weighs heavily in my 

22    mind, although it does not appear in any of the loss 

23    function analyses that I have seen, is that the population 

24    ratios, for example, the population of A to B -- let me just 

25    do that one.  In the original enumeration, the ratio of A to 
                                                              2540

 1    B in terms of population is two to one.  Adjustment makes it 

 2    perfect.  The most dramatic example, of course, is A to E.  

 3    In the original enumeration, which is like the census, the 

 4    ratio is infinite, 2 divided by zero.  I have never before 

 5    testified about infinity.  

 6               The adjustment moves from a ratio of 2 to zero to 

 7    a ratio of 2 to 1, which is a lot closer to truth, which is 

 8    2 to 2.  

 9               So to me these data would favor adjustment.  I 

10    don't remember this hypothetical as particularly realistic.  

11    We do know truth here, and we are adjusting an area which 

12    has zero population.  So at least in these two respects I 

13    think this hypothetical is unrealistic.  But if these were 

14    the facts, I would favor adjustment on that basis.

15         Q.    When you say "unrealistic," what do you mean 

16    unrealistic in relation to?

17         A.    They are different.  As I see it, it is the 

18    problem of trying to decide whether or not to adjust the 

19    census of 1990 using the production dual-system estimates. 

20         Q.    Dr. Freedman, Mr. Solomon asked you some 

21    questions about the formula that appears, I believe, on page 

22    35 of Defendants' Exhibit 64.

23         A.    Right.

24         Q.    He took you through a number of permutations and 

25    asked you as part of his questions to consider dividing G.  
                                                              2541

 1    Do you recall that?

 2         A.    Yes.

 3         Q.    You testified that you were unwilling to consider 

 4    dividing G?

 5         A.    I was willing to consider it, but I wasn't 

 6    willing to change my formula.

 7         Q.    Why are you not willing to divide G in your 

 8    formula on page 35?

 9         A.    What I am trying to get at by this technique is 

10    the relationship between truth, represented by mu, and the 

11    production dual-system estimator, represented by X.  I am 

12    trying to describe the joint properties of truth and the 

13    estimate for it.  

14               What Mr. Solomon seems to want to do is to try 

15    and consider the relationship between truth or half of truth 

16    and half the dual-system estimator.  I am not interested in 

17    half the dual-system estimator.  I am interested in the 

18    dual-system estimator itself.  

19               MR. SITCOV:  No further questions, your Honor.  

20               THE COURT:  Thank you.  Can we excuse Dr. 

21    Freedman?  

22               MR. SOLOMON:  Your Honor, we can.  I want the 

23    documents that we didn't have that inhibited 

24    cross-examination.  I don't think there is going to be a 

25    need to see Dr. Freedman in this context anymore.  Subject 
                                                              2542

 1    to that, I have nothing further.  

 2               THE COURT:  Thank you very much, Dr. Freedman.  

 3    Most illuminating.  

 4               (Witness excused) 

 5               MR. SITCOV:  Your Honor, at this time I would 
                                                               

 6    like to move into evidence Defendants' Exhibit 67.  It was 

 7    something that was discussed by Dr. Ericksen during his 

 8    cross-examination.  It is in the binder of defendants' 

 9    exhibits.  

10               THE COURT:  67 is entitled "Regression Models for 

11    Adjusting the 1980 Census" by Freedman and Navidi.  

12               MR. SITCOV:  Yes.  

13               MR. SOLOMON:  We object.  In relation to the 1980 

14    census, Professor Freedman was on the stand.  He could have 

15    asked him any questions he wanted to about that.  It is 

16    hearsay.  It is completely irrelevant.  

17               MR. SITCOV:  Dr. Ericksen testified rather 

18    extensively about it on cross-examination, your Honor, and 

19    identified, for example -- 

20               THE COURT:  I don't recall this.  Why did you 

21    wait till now to offer it?  

22               MR. SITCOV:  I think that I probably just forgot 

23    to or it at the time, your Honor.  

24               THE COURT:  Ah.  And it was brought out during 

25    Dr. Ericksen's -- 
                                                              2543

 1               MR. SITCOV:  Yes, it was.  

 2               THE COURT:  He was the first witness.  

 3               MR. SITCOV:  Yes, he was.  I apologize for my 

 4    mistake.  

 5               MR. SOLOMON:  Your Honor, it was brought out on 

 6    cross.  Mr. Sitcov got whatever he wanted out of the cross- 

 7    examination.  We have this document marked for 

 8    identification.  There is not going to be any question about 

 9    what it is.  But, respectfully, it should not be evidence.  

10               THE COURT:  I will admit it.  Defendants' Exhibit 

11    67 is admitted. 

12               (Defendants' Exhibit 67 for identification was 

13    received in evidence) 

14               MR. SOLOMON:  Just a point of clarification.  It 

15    is that document that includes comments, including that of 

16    Professor Madansky?  

17               MR. SITCOV:  Yes.  

18               MR. SOLOMON:  You are offering the whole 

19    document?  

20               MR. SITCOV:  Yes.  

21               MR. SOLOMON:  Thank you.  

22               MR. SITCOV:  Also, at this time, your Honor, 

23    defendants wish to renew their motion to exclude certain 

24    charts that were introduced through the testimony of 

25    Franklin Fisher and to exclude his testimony with respect to 
                                                              2544

 1    those charts.  I know your Honor has the dueling 

 2    declarations that refer to that.  In our opinion, there 

 3    isn't any doubt but that plaintiffs had had everything they 

 4    needed in order to compute the charts as of the end of 

 5    February of this year.  

 6               I think perhaps one of the more striking 

 7    demonstrations of that, as we pointed out in our 

 8    declarations, is that we have received two different 

 9    versions of these charts from the plaintiffs.  One we 

10    received on the eve of trial.  One we received at trial.  

11    And there are certain rather significant differences in the 

12    exhibits.  

13               For example if you will look at 684, 685, 686, 

14    687'.  I'm sorry.  I think your clerk has a copy of the 

15    papers that we have submitted.  

16               If your Honor would take a look at attachment A, 

17    you will see Plaintiffs' Exhibit 684 and 686.  If you look 

18    at the bottom of those two documents, there is, I guess, 

19    some sort of citation that simply says "PRODSE."  

20               If you look at attachment B, which reproduces the 

21    same -- well, those are supposed to be the same exhibits.  

22    Now they have, under PRODSE, a number of data files 

23    represented.  You will find the same thing if you compare 

24    the version of 686, that is at attachment A, with the 

25    version of 686 in attachment B. 
                                                              2545

 1               Another perhaps even more interesting difference 

 2    would be comparing I think it is 696, the two versions of 

 3    696.  If you compare the 696 that appears as attachment A 

 4    with the 696 that appears at attachment B, in the bold-face 

 5    type at the top of the large squared area, it says number of 

 6    standard deviations from zero, 5.89.  That is the 696 that 

 7    we got on the eve of trial.  If you look at the version of 

 8    696 that the plaintiffs submitted in court, you will see it 

 9    gives a rather different number.  That number is 584.  

10               THE COURT:  524?  

11               MR. SITCOV:  I'm sorry.  524.  I take that back.  

12               Again, if you look at the sourcing materials, the 

13    698 that is at Attachment A simply says PRODSE final loss 

14    function analyses.  696 that is in Attachment B, it lists I 

15    think the same data files that are listed for the other 

16    things in Attachment B.  

17               We have seen two versions of these documents with 

18    no explanation as to why we have seen two versions and 

19    certainly no notice to us that we had two versions.  Had 

20    someone not actually gone through and pulled them out, we 

21    would never have found that out. 

22               The other thing, and I think it is certainly 

23    significant, the 696 document that I was just referring to 

24    in both versions it has the number 524 is a certain type of 

25    statistic that is suggestive of a strongly estimated number.  
                                                              2546

 1    During Dr. Fisher's deposition, he testified about that 

 2    number and that he knew what that number was.  So clearly 

 3    the plaintiffs had in their possession at this time, at the 

 4    time of his deposition, which was April 15th, 16th, and 

 5    17th, the data necessary to derive this number.  

 6               I think it is clear from the Kortakowsky and the 

 7    Lyon declarations and even, I would say, from the 

 8    declarations that were submitted by the plaintiffs, that all 

 9    the material that they needed to compute the data that 

10    appears on these documents was made available to them before 

11    the end of February of this year.  

12               There is absolutely no reason why they can't have 

13    done this in time enough for their witnesses to know about 

14    it in depositions and for us to be able to inquire of them 

15    during depositions on these topics.  In fact, as I said, Dr. 

16    Fisher apparently knew about what that T statistic on 696 

17    was going to be, because he testified that the number was 

18    about 5.  

19               So it is simply incredible to me that the 

20    plaintiffs might say that they didn't have the information 

21    necessary to make these charts up.  When you take that, your 

22    Honor, in connection with the differences in the two 

23    versions of these exhibits with no explanation as to the 

24    difference, we find that quite problematic and certainly 

25    prejudicial.  
                                                              2547

 1               THE COURT:  Mr. Rifkind? 

 2               MR. RIFKIND:  First let me say about the dueling 

 3    affidavits.  They pass each other in the night.  There is a 

 4    long discussion in the affidavit.  But it doesn't address 

 5    the proposition asserted in the affidavit submitted by Brian 

 6    Palmer, which is attached to Mr. Bowker's affidavit, that 

 7    CRA was not able to replicate the final loss function 

 8    analysis as of the middle of March because we possessed 

 9    neither the programs used by the Bureau to perform the final 

10    loss function analysis nor the algorithm employed by the 

11    Bureau for that purpose.  One or the other of those was 

12    necessary to replicate the final loss function analysis.  

13               Then Mr. Palmer goes on to describe how, by a 

14    process of detective work, in April they found a document 

15    which suggested a method by which they might replicate it, 

16    cracked the code, developed their own version of it, which 

17    later turned out to be identical to the Bureau's or 

18    substantially identical to the Bureau's, and that therefore 

19    the analysis of the loss function data could not have been 

20    performed until sometime after, I believe it was, April 

21    23rd, as stated in -- 

22               THE COURT:  I don't think I need much more 

23    guidance on that point.  What about the fact that you have 

24    two sets of books here?  

25               MR. RIFKIND:  The charts, there are two types of 
                                                              2548

 1    changes.  In the version we gave them on the 1st of May, in 

 2    the process of preparing these in white heat, having only 

 3    received the calculations at the very end of April, we 

 4    omitted to put on the chart, as we had intended to do, the 

 5    data sources, the data files, the Bureau data files from 

 6    which the calculations are derived.  So for most of these 

 7    charts the difference is that we added one line there 

 8    identifying the file source.  

 9               That information was available to them on May 1.  

10    Indeed, we asked them on May 1, is there anything else you 

11    want to know?  But we thought for clarity of preparation it 

12    ought to be on the physical chart when we presented it in 

13    court, and we added that.  And we called that to their 

14    attention some days before Dr. Fisher testified.  

15               As Dr. Fisher also testified on the stand, on 

16    direct, he found that someone had miscounted the number of 

17    zeros in one of these calculations, missed a zero, and that 

18    changed in a modest way the number of standard deviations 

19    calculation and a few of the other calculations.  The 

20    difference between the chances of something being true are 

21    one in 80 million or one in 3 trillion doesn't seem to me 

22    exactly earthshaking.  But on direct examination Dr. Fisher 

23    pointed that out.  They were free to cross-examine him at 

24    the time.  

25               THE COURT:  Why do you need both sets of 
                                                              2549

 1    documents?  

 2               MR. RIFKIND:  No.  We only want the ones we 

 3    offered in evidence and were received in evidence, and those 

 4    were the corrected ones.  They would have corrected it by 

 5    pen from the stand if we hadn't had it printed up in 

 6    advance.  

 7               THE COURT:  I misunderstand what Mr. Sitcov was 

 8    saying.  I thought he was saying both were being offered.  

 9               MR. RIFKIND:  No.  

10               MR. SITCOV:  No.  We got both.  

11               MR. RIFKIND:  We gave them one on May 1.  Then 

12    about three days before Dr. Fisher testified, we found the 

13    error and SENT them that I think three days before.  I think 

14    Mr. Sitcov's indication of what Dr. Fisher testified to at 

15    his deposition is not borne out by the deposition testimony.  

16    Whether he had some expectation of what the values would be 

17    or not, perhaps the deposition indicates.  But the fact is 

18    he hadn't done the calculation.  That is only done following 

19    the 23rd of April, and he couldn't have done it before that.

20               THE COURT:  I think I am ready to rule.  

21               MR. RIFKIND:  I might add one last point on this.  

22    The dramatic difference between the situation faced with 

23    respect to Freedman and the situation faced with respect to 

24    Dr. Fisher is quite clear.  We still are waiting for the 

25    data with respect to Dr. Freedman.  
                                                              2550

 1               MR. SITCOV:  May I make just one more comment?  

 2               THE COURT:  Yes.  

 3               MR. SITCOV:  Dr. Fisher testified, and I would be 

 4    happy to go across the hall and get his deposition and read 

 5    it, Dr. Fisher testified that someone working on plaintiffs' 

 6    behalf had already as of April 15th of this year computed 

 7    the T statistic that appears in Plaintiffs' Exhibit 696.  

 8    That is simply impossible if they didn't have the data that 

 9    were required to produce those charts.  

10               THE COURT:  Thank you.  

11               The plaintiffs have offered Exhibits 684, 686, 

12    and 696 through and including 701.  Defendants object to the 

13    admission of all of the defendants.  

14               There are basically two grounds.  First, some of 

15    the documents appear in duplicate and they are not the same.  

16    That has been resolved by the plaintiffs offering only the 

17    later of the two documents.  So my ruling pertains only to 

18    the admissibility of the later of the documents.  

19               The other ground is that the plaintiffs and 

20    defendants have a serious factual dispute as to whether the 

21    calculations represented in these documents could have been 

22    made before April 23.  The defendants say yes, the 

23    plaintiffs say no.  Each has put in affidavits supporting 

24    its side of the case.  

25               I am obviously not in a good position to decide 
                                                              2551

 1    that issue of fact.  Accordingly, I ought to err, if I am 

 2    going to, on the side of admissibility.  I will therefore 

 3    admit all of those documents, giving appropriate 

 4    consideration to the arguments raised by the defendents that 

 5    the documents might have been compiled before April 23. 

 6               (Plaintiffs' Exhibits 684, 686, and 696 through 

 7    701 for identification were received in evidence) 

 8               THE COURT:  Anything else?  

 9               MR. SOLOMON:  Just one thing, your Honor.  I 

10    would like to mark that chart for identification purposes as 

11    the next PX number and offer it merely as what we used 

12    during the cross-examination.  

13               THE COURT:  I will admit it for that purpose.  If 

14    you can give us a number.  

15               MR. RIFKIND:  767.  

16               THE COURT:  767 is admitted for illustration 

17    purposes as an adjunct to the cross-examination of Dr. 

18    Freedman. 

19               (Plaintiffs' Exhibit 767 for identification was 

20    received in evidence) 

21               MR. RIFKIND:  I have a further housekeeping 

22    matter.  We have discussed putting in the record various 

23    dispositions.  I wonder whether this isn't the appropriate 

24    time to do that.  

25               THE COURT:  Yes, I remember Mr. Cohen.  I see him 
                                                              2552

 1    coming at me.  Even if your name is Quinn.  

 2               MR. COHEN:  I was suggesting to somebody this 

 3    morning, your Honor, that if this were the Army, I would be 

 4    the person handed the toothbrush to clean the floors.  

 5               MR. MILLET:  Your Honor, I had understood from 

 6    our earlier side bar that this was going to be done after 

 7    the formal close of the proceedings.  They have designated 

 8    additional transcript pages outside the schedule that we 

 9    have done pretrial.  I haven't had a chance to look them up.  

10               MR. COHEN:  I don't plan to belabor this.  I just 

11    want to tell the Court what it is that we are submitting.  

12    Obviously, defendants will have the time to -- 

13               THE COURT:  I wonder if it isn't more efficient 

14    to do the whole thing on paper.  

15               MR. MILLET:  That is our suggestion.  

16               MR. COHEN:  It is essentially paper which I am 

17    going to hand up to you, your Honor.  

18               THE COURT:  They haven't looked at everything.  

19    Why don't we wait until they do and then we will get a 

20    complete set of papers from each side, each of you with your 

21    designations and cross-designations and objections.  I will 

22    look at them and issue an order telling which are in and 

23    which are out.  

24               MR. COHEN:  That is fine.  There is one other 

25    housekeeping matter, your Honor.  There has been a pending 
                                                              2553

 1    motion by the State of Oklahoma to bring witnesses.  We 

 2    managed to resolve that.  Oklahoma has provided us and 

 3    defendants affidavits.  Both sides have agreed that were the 

 4    witnesses to appear, the affidavits represent what the 

 5    testimony would be.  We don't, obviously, stipulate that the 

 6    facts are the truth.  

 7               I am authorized by counsel for Oklahoma to state 

 8    that they have withdrawn their motion seeking to present 

 9    live testimony.  

10               THE COURT:  I will accept the affidavits that you 

11    have just handed up.  

12               MR. RIFKIND:  There is one further matter that 

13    might be raised at this time.  Mr. Segen for Hudson County 

14    wishes to be heard.  Perhaps we could deal with that next.  

15               THE COURT:  Mr. Segen.  

16               MR. SEGEN:  Good afternoon, Judge McLaughlin.  I 

17    would like to say that I find it somewhat apt that I should 

18    close the proceedings this morning, but I am not sure that 

19    that is the case.  

20               When I was last here, the issue was the voting 

21    rights act and which of the possibility of 175 different 

22    methods that were brooded about to fill the gap in regard to 

23    the evidence on the Voting Rights Act case would ultimately 

24    be employed.  I am glad to be able to tell you, your Honor, 

25    that as a result of help from your law clerk Sean McGrath 
                                                              2554

 1    and Dave Goldii of the plaintiffs' committee and Mr. Millet 

 2    representing the government, that we have, in fact, opted 

 3    for a solution, albeit one that I would not have thought was 

 4    going to be the one to be presented.  

 5               It proceeds in a two-step approach, and it 

 6    proceeds from the basic fact that it is the government's 

 7    position that the Voting Rights Act does not, in fact, apply 

 8    to the federal government.  The approach that the government 

 9    has accepted, defendants have accepted, that in the first 

10    instance, if it is acceptable to the Court, the Court will 

11    evaluate the validity of Hudson County's claim, legal claim, 

12    that in fact the Voting Rights Act does apply to the 

13    government.  

14               If the Court decides that it does, Hudson County 

15    will then proceed with affidavits from the appropriate 

16    expert to fill in the gap in regard to the impact of using 

17    adjusted statistics on minority districts.  It is my 

18    understanding that defendants would have full discovery 

19    available to them in that regard.  I believe they will be 

20    making that application after I finish.  

21               THE COURT:  Discovery as to what?  

22               MR. SEGEN:  In regard to that affidavit, in 

23    regard to that witness.  

24               THE COURT:  I don't understand what that means.  

25               MR. MILLET:  Your Honor, I think what we are 
                                                              2555

 1    basically talking about is something akin to summary 

 2    judgment motions.  They will brief the legal issue.  In they 

 3    think they need some facts to support it, they will put it 

 4    in an affidavit.  We will oppose it if we think it is a 

 5    legal issue.  If we want to oppose it about an opposing 

 6    affidavit, we will do it.  If we think we need to take 

 7    discovery, we will do that.  

 8               THE COURT:  I understand.  All right.  

 9               MR. SEGEN:  Your Honor, I just would add that at 

10    this point Hudson County, in lieu of the affidavits, would 

11    provide the Court with an offer of proof as to what in 

12    general the affidavit would contain.  If Hudson County were 

13    to put on proof that would establish that the use of the 

14    uncorrected figures rather than adjusted statistics results 

15    in violations of the Voting Rights Act on the federal, 

16    state, and municipal levels, and that the undercount of 

17    minorities that results from this determination deprives a 

18    geographically compact cohesive minority group that votes as 

19    a block in a number of minority districts -- no, excuse me 

20    -- that votes as a block:  One, that there are violations in 

21    a number of minority districts that would result in adding 

22    minority districts; and, 2, also results in packing that 

23    requires reconfiguration of a number of minority districts, 

24    thereby releasing members of that compact cohesive minority 

25    group from a packed minority district into another district.  
                                                              2556

 1               THE COURT:  Thank you.  

 2               MR. SEGEN:  That would occur in a number of 

 3    states.  Thank you, your Honor.  

 4               THE COURT:  Thank you, Mr. Segen.  

 5               MR. SITCOV:  I do have one other thing, your 

 6    Honor.  Since Mr. Solomon questioned Dr. Freedman upon 

 7    Defendants' Exhibit 63, we would like that to be admitted 

 8    into evidence.  

 9               MR. SOLOMON:  What is 63?  

10               THE COURT:  63 or 163?  

11               MR. SITCOV:  63.  

12               MR. SOLOMON:  Your Honor, that is the same 

13    document that you already ruled on.  I questioned the 

14    witness on cross-examination for the basis of the opinions 

15    that he expressed concerning it.  I didn't offer it.  We 

16    object and continue to object on all grounds. 

17               THE COURT:  I will exclude it.  

18               MR. MILLET:  If everybody else is done, we rest.  

19               THE COURT:  Very good.  

20               MR. RIFKIND:  Your Honor, if it is an appropriate 

21    time to break for lunch, we would like to consider over 

22    lunch what, if anything, we would propose to put on in the 

23    way of rebuttal.  

24               THE COURT:  This afternoon?  

25               MR. RIFKIND:  This afternoon.  
                                                              2557

 1               THE COURT:  Do you want to come back at two?  

 2               MR. RIFKIND:  Could we have maybe just a little 

 3    bit longer?  

 4               THE COURT:  Yes.  2:30, 4:30, pick it.  

 5               MR. RIFKIND:  2:30 will be terrific.  

 6               THE COURT:  We will resume at 2:30.  

 7               (Luncheon recess) 

 8    

 9    

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2558

 1               A F T E R N O O N     S E S S I O N.

 2                                      2:30 o'clock p.m.  

 3               MR. RIFKIND:  Your Honor, we propose, if this is 

 4    satisfactory to your Honor, to call a witness this afternoon 

 5    in rebuttal, Dr. Rolph.

 6               I did not, when we made our plans, appreciate the 

 7    court was going to be available tomorrow and I understand --  

 8               THE COURT:  I didn't plan to be, I'm just trying 

 9    to be accommodating.  

10               MR. RIFKIND:  I appreciate the accommodation.  
                                                                

11    Unfortunately, I told my experts that they could go away for 

12    the weekend about three weeks ago and some of them did.

13               On the other hand, I'm hopeful to have a witness 

14    here tomorrow, in fact, I'm hopeful to have Dr. Fisher back, 

15    and what I advised Mr. Sitcov and your clerk is that I would 

16    advise both of them tomorrow morning if we have succeeded in 

17    getting Dr. Fisher back, and in that case we would propose 

18    to proceed as soon as you are available tomorrow afternoon.  

19               THE COURT:  All right.  

20               THE COURT:  Is that all for today?  

21               MR. RIFKIND:  Dr. Rolph will be examined by Mr. 

22    Sherman.  

23               MR. SITCOV:  I wonder if we can't find out today 

24    if Dr. Fisher is available tomorrow?  

25               THE COURT:  When will you be able to determine 
                                                              2559

 1    today whether Dr. Fisher will be available tomorrow?  

 2               MR. RIFKIND:  There are really two questions 

 3    related, one, is he available and can I get him down here, 

 4    and I hope to know that answer this evening.

 5               The second question is whether after we review 

 6    the transcript of the testimony of yesterday afternoon and 

 7    this morning it is worth your Honor's while and ours to put 

 8    him on.  

 9               THE COURT:  And that, I presume, will take until 

10    at the earliest sometime tonight?  

11               MR. RIFKIND:  That will take some time tonight 

12    and we get the transcript in the early evening and it will 

13    NOT be UNTIL tomorrow morning.

14               I am not going to do it if it is A cumulative 

15    things, your Honor heard plenty.  

16               MR. SITCOV:  We would like to know.  

17               THE COURT:  As soon as they decide, give them a 

18    telephone number and they will call you.  

19               MR. SITCOV:  Great.  

20               THE COURT:  Dr. Rolph, come on up here.

21    JOHN ROLPH,

22               called as a witness in rebuttal by the plaintiff,

23               was examined and testified as follows:

24               THE COURT:  I won't bother to swear you again, I 

25    think the original oath still holds. 
                                                              2560

 1               THE WITNESS:  Okay.

 2               Mr. Sherman.  

 3               MR. SHERMAN:  Good afternoon, your Honor.  

 4    DIRECT EXAMINATION 

 5    BY MR. SHERMAN:

 6         Q.    Dr. Rolph, were you in court yesterday and today 
                                                                  

 7    for Dr. Freedman's testimony?

 8         A.    Yes, I was.

 9         Q.    And have you read his two papers on smoothing?

10         A.    Yes, I have.

11         Q.    Did you hear Dr. Freedman testifying that 

12    modeling of variance is not a standard procedure?

13         A.    Yes, I did.

14         Q.    Do you know if the Bureau of the Census models 

15    variances in other areas?

16         A.    Yes, I believe they do.

17         Q.    Did you hear Dr. Freedman testifying about 

18    benchmarking or ratio adjustment?

19         A.    Yes, I did.

20         Q.    Do you know if the Bureau of the Census uses 

21    ratio adjustment or benchmarking in other contexts?  

22         A.    Yes, they do.  Like modeling variances, I think 

23    it is routinely used, for example, in the current population 

24    survey.  

25               MR. SHERMAN:  May I approach the witness, your 
                                                              2561

 1    Honor?  

 2               THE COURT:  Yes.

 3         Q.    Dr. Rolph, I have handed you what I have marked 

 4    as Plaintiff's Exhibit 763.

 5               Can you tell the court what this is?

 6         A.    Yes.  This is one of the reports from the current 

 7    population survey, one of the recent ones, actually.  It's 

 8    entitled, "Poverty in the United States: 1990," series P 60 

 9    No. 175.

10         Q.    Is this a document from the Census Bureau?

11         A.    Yes, it is.

12         Q.    And does this document reflect modeling of 

13    variances?

14         A.    Yes, it does.  It has a section later on in the 

15    document I believe on what looks to be page 208, which gives 

16    a formula which indicates that there is what amounts to a 

17    linear combination of two terms used in modeling variances.

18         Q.    Does this document indicate that the Bureau did 

19    ratio adjustment in conjunction with the modeling of 

20    variances?

21         A.    Yes, it also indicates that.  

22               MR. SHERMAN:  Your Honor, we would move 

23    Plaintiff's Exhibit 763.  

24               MR. SITCOV:  No objection.  

25               THE COURT:  763 is admitted.
                                                              2562

 1               (Plaintiff's Exhibit 763 marked for 

 2    identification was received in evidence.) 

 3    BY MR. SHERMAN:

 4         Q.    I'm sorry, Dr. Rolph, is there a difference 

 5    between 763 and 764?

 6         A.    Yes.  764 is another example of the use of 

 7    modeling variances and ratio adjustment for the current 

 8    population survey.  It's titled, "Household and family 

 9    characteristics: March 1991." 

10               It also, as did the previous document, indicates 

11    that variances are modeled and ratio adjustment is used.

12         Q.    And is it a Bureau of the Census document?

13         A.    Yes, it is put out by the Bureau of the Census.  

14               MR. SHERMAN:  Your Honor, we would move 764.  

15               MR. SITCOV:  No objection.  

16               THE COURT:  764 is admitted.

17               (Plaintiff's Exhibit 764 marked for 

18    identification was received in evidence.) 

19    BY MR. SHERMAN:

20         Q.    Have you had a chance to look at that, Dr. Rolph?

21         A.    Yes, I have.

22         Q.    I have handed you what has been marked as 

23    Plaintiff's Exhibit 768.

24               Will you tell the court what that is?

25         A.    Yes.  This is the underlying, if you will, 
                                                              2563

 1    technical --  I don't want to call it an appendix, but a 

 2    description of how to do ratio adjustment.

 3               As you can see from the front of it, it was put 

 4    out some time ago in 1963.  Its technical paper number 7 

 5    entitled, "The current population survey, a report on 

 6    methodology," and the portion that is in the exhibit is part 

 7    7 entitled, "Preparation of estimates," and it describes in 

 8    some detail how to do ratio estimation and it's the method 

 9    that is used for the current population survey.

10         Q.    Does it also involve modeling of variances?

11         A.    This particular exhibit doesn't.  I believe there 

12    is another chapter in the same report that does modeling of 

13    variances.  

14               MR. SHERMAN:  Your Honor, we would move 768.  

15               MR. SITCOV:  No objection.  

16               THE COURT:  768 is admitted.

17               (Plaintiff's Exhibit 768 marked for 

18    identification was received in evidence.) 

19    BY MR. SHERMAN:

20         Q.    Dr. Rolph, I have handed you what are marked as 

21    Plaintiff's Exhibits 769 and 770.

22               Will you tell the court what these are?

23         A.    Yes.  These are two Bureau of the Census 

24    publications, or I beg your pardon, these are two chapters 

25    in the Bureau of The census publication entitled, "The 
                                                              2564

 1    current population survey, design and methodology."

 2               769 is chapter 8, 770 is chapter 5.

 3               Chapter 8, which is 769, describes modeling of 

 4    the variances in the presence of a replication method that 

 5    is very much like the jackknife that we talked about 

 6    earlier.

 7               770 lays out in more detail the use of ratio 

 8    estimates in the current population survey and is basically 

 9    the methodological description of how it is done.

10         Q.    Dr. Rolph, we have used the terms ratio 

11    adjustment and benchmarking interchangeably.

12               Are they the same thing?

13         A.    As I understand the way Dr. Freedman used them in 

14    his testimony, he used benchmarking as a synonym for ratio 

15    adjustment, he also used the term, I believe, scaling 

16    variances as a synonym for ratio adjustment of the 

17    variances.

18         Q.    Do the terms presmoothing and modeling variances, 

19    to your knowledge, refer to the same thing?

20         A.    Yes.  Modeling variances would be the standard 

21    usage in the profession.  I think presmoothing has become to 

22    some use standard usage in this case.  

23               MR. SHERMAN:  Your Honor, at this time plaintiffs 

24    would move 769 and 770.  

25               MR. SITCOV:  No objection.  
                                                              2565

 1               THE COURT:  Both are admitted.

 2               (Plaintiff's Exhibits 769 and 770, respectively, 

 3    marked for identification were received in evidence.)

 4    BY MR. SHERMAN:

 5         Q.    Dr. Rolph, did you hear Dr. Freedman testify that 

 6    the assumptions underlying the smoothing model do not hold 

 7    precisely?

 8         A.    Yes, I did.

 9         Q.    I would like to go through the assumptions that 

10    he talked about in his testimony.

11               The first one was about bias in the dual system 

12    estimation, correct?

13         A.    Yes, that was one of his assumptions.

14         Q.    And what did Dr. Freedman say about the 

15    assumption about --  first of all, what is the Bureau's 

16    assumption underlying the smoothing model about bias in dual 

17    system estimation?

18         A.    The literal assumption is that there is no bias 

19    in the dual system estimate.

20         Q.    And what did Dr. Freedman testify to about that 

21    assumption?

22         A.    He indicated, as I recall, that he believed that 

23    it was incorrect.

24         Q.    And, Dr. Rolph, is there bias in the dual system 

25    estimation?
                                                              2566

 1         A.    Yes, I believe there is.

 2         Q.    How do you know that?

 3         A.    Well, the Census Bureau studied this problem and, 

 4    indeed, I believe it's P16, if I remember the numbering 

 5    correctly, and they did a study in their total error model 

 6    looking at, among other things, the bias of the dual system 

 7    estimate of the raw dual system estimate as well as the 

 8    smoothed dual system estimate.

 9         Q.    Were the results of that study available to the 

10    undercount steering committee last June?

11         A.    Yes, my understanding is that they were available 

12    and they were discussed.

13         Q.    And so did the undercount steering committee last 

14    June know that there was some departure from that 

15    assumption?  

16               MR. SITCOV:  I object.  

17               THE COURT:  Overruled.

18         A.    Yes, according to my reading of the 

19    documentation, they did.

20         Q.    And whatever Dr. Freedman's opinion about the 

21    effect of bias in that assumption, the undercount steering 

22    committee did not reject the smoothing model because of it, 

23    did they?

24         A.    No.  Indeed, if I am recalling correctly, the 

25    estimates from the total error model of bias in the dual 
                                                              2567

 1    system estimate I believe on net was slightly smaller when 

 2    it was smoothed as opposed to when it was not smoothed, but 

 3    there was some variation, depending on which evaluation 

 4    post-strata you looked at.

 5         Q.    Did Dr. Freedman's testimony provide any new 

 6    evidence about bias in the dual system estimate?

 7         A.    Not as I heard it, no.

 8         Q.    Did his papers provide any new evidence?

 9         A.    Not as I read them, no.

10         Q.    Do you remember the second assumption that Dr. 

11    Freedman talked about with regard to the smoothing model?

12         A.    The famous sigma squared I and V 3?  Yes, I do.

13         Q.    Famous might not be the word used in some 

14    quarters.

15         A.    Infamous?

16         Q.    Is it true, Dr. Rolph, that that equation meant 

17    that the Bureau's smoothing model took no account of 

18    correlations between post-strata?

19         A.    Not exactly.  It was an assumption about the 

20    relationship of the true underlying adjustment factor to 

21    this regression.  The Bureau's model also included sampling 

22    error and in the sampling error they allowed for correlation 

23    across post-strata.

24         Q.    So another point in the process they did take 

25    into account correlations between post-strata?
                                                              2568

 1         A.    That's correct, in the sampling error.

 2         Q.    So it was just in this particular point in the 

 3    process, the equation which represents the variance of truth 

 4    around regression surface, that the Bureau chose not to 

 5    build in correlations between post-strata?

 6         A.    I think that's an accurate statement, yes.

 7         Q.    Are you familiar with what you call Dr. 

 8    Freedman's famous alternative?

 9         A.    Yes, I believe I was asked about it at my 

10    deposition.

11         Q.    Do you know why the Bureau chose to use sigma 

12    squared I?

13         A.    Well, my understanding is that they did several 

14    studies looking at departures from the sigma squared I 

15    assumption both in analyzing the 1988 dress rehearsal data 

16    as well as in the 1990 census.

17         Q.    Is it your view that sigma squared I is a more 

18    appropriate term to use than Dr. Freedman's alternative?

19         A.    Yes.  I base that on really two pieces of 

20    information:

21               The first one is that the Bureau did several 

22    studies looking at sigma squared I versus other assumptions, 

23    some of which actually were fairly close to the V 3 

24    assumption, and, secondly, I find the V 3 assumption on its 

25    face somewhat unreasonable for the following reason:
                                                              2569

 1               Basically, the idea behind smoothing is to try to 

 2    reduce the effect of sampling error.  The V 3 assumption 

 3    will take a post-strata which had a very small sampling 

 4    error and move it towards the regression surface by the same 

 5    amount as a post-strata that has an enormous sampling error, 

 6    and that strikes me as not being a terribly reasonable thing 

 7    to do statistically.

 8         Q.    Did you hear Dr. Freedman testify that the Bureau 

 9    did not consider the extent to which the assumptions of that 

10    equation held?

11         A.    I recollect him testifying along those lines, 

12    yes.

13         Q.    Is it correct in your view that the Bureau did 

14    not consider the extent to which that assumption held?

15         A.    No.  As I indicated, I think they did a number of 

16    studies looking specifically at that assumption.  

17               MR. SHERMAN:  May I approach the witness, your 

18    Honor?  

19               THE COURT:  Sure.

20               (Handing to the witness)

21               (Pause) 

22    BY MR. SHERMAN:

23         Q.    Are you familiar with this document, Dr. Rolph?

24         A.    Yes, I am.

25         Q.    Can you describe to the court what it is?
                                                              2570

 1         A.    Yes.  It's entitled, "Memorandum for the 

 2    technical design and estimation committee," dated January 2, 

 3    1990 from Greg Diffendal, and this memorandum is a brief 

 4    description of some work done by Cary Isaki and done by Bill 

 5    Bell comparing the sigma squared I assumption to two other 

 6    assumptions.

 7         Q.    Without getting too technical, can you tell us 

 8    what the other assumptions are?

 9         A.    As I understand them from this document and 

10    others, one of them was using rather than the identity 

11    matrix using the correlation, the sampling correlation 

12    matrix, and the other one was using instead of V 3 what is 

13    demoted V zero, which is I guess I would call it a slight 

14    variant of V 3 is probably the simplest way to say it.

15         Q.    Does this document represent a test of the 

16    assumption represented by sigma squared I?

17         A.    That's my understanding of the work that was done 

18    there.  

19               MR. SHERMAN:  Your Honor, we would move 

20    Plaintiff's Exhibit 70.  

21               MR. SITCOV:  I object.  I move to strike.  This 

22    is all hearsay.  We can't rely on the information in here.  

23    He has no knowledge whatsoever of these things.  He hasn't 

24    testified that he knows about these things, he is just 

25    telling what he thinks based on reading this.
                                                              2571

 1               This is not his testimony about what the Bureau 

 2    did.  

 3               THE COURT:  Mr. Sherman?  

 4               MR. SHERMAN:  I am unclear whether Mr. Sitcov is 

 5    objecting to the document, which has a Bates stamp which 

 6    appears to indicate that it is in the administrative record. 

 7               Regardless of that, we would say it should be 

 8    admitted for the Bureau's state of mind.

 9               It contradicts the testimony of one of 

10    defendants' experts.  

11               MR. SITCOV:  The Bureau certainly doesn't have a 

12    state of mind.  

13               THE COURT:  That is very clear.  

14               MR. SITCOV:  If the document is in the 

15    administrative record that is one thing, but --  

16               THE COURT:  Is it?  Is that --  

17               MR. SITCOV:  I can't tell if it's ours or not.  

18    There are several different Bates stampes I have seen 

19    bandied about, and I can't tell you whether this particular 

20    one is ours.  

21               THE COURT:  I assume it is.  I will take it in on 

22    that basis.  If it is wrong, you let my know later on.  

23               MR. SITCOV:  I would still like to strike the 

24    testimony of this witness as to what the Bureau is doing.  

25               THE COURT:  The testimony can stand as he simply 
                                                              2572

 1    interpreted this document.  Without the documents it can't 

 2    come in, but with the document it's all right.

 3               (Plaintiff's Exhibit 70 marked for identification 

 4    was received in evidence.) 

 5    BY MR. SHERMAN:

 6         Q.    Dr. Rolph, I have handed you PX 87.

 7               Would you describe to the court what this is?

 8         A.    Yes.  It's entitled at the top, "A draft, C.T. 

 9    Isaki, 11/15/90."  The title of the document is, "Smoothed 

10    factors using sigma squared V sub zero."  

11               THE COURT:  Zero? 

12               THE WITNESS:  Zero.  That's my interpretation of 

13    the blob on my Xerox.

14         Q.    What does this document represent?

15         A.    This represents, again, testing a departure from 

16    the sigma squared I assumption and replacing it with sigma 

17    squared V sub zero, and it, again, is another example, but 

18    it's somewhat greater length of the document we were just 

19    discussing, in the Bureau's analysis of alternative 

20    assumptions.

21         Q.    Is V sub zero similar to V sub 3?

22         A.    Yes.  They define it toward the end of the first 

23    paragraph.  It states here, "V sub zero is V sub 3, but 

24    absent the P-sample component of V sub 3."

25         Q.    In your view, does this document show that the 
                                                              2573

 1    Bureau considered alternatives to sigma squared I?

 2         A.    Yes, that's exactly what it shows.  

 3               MR. SHERMAN:  Your Honor, we would move PX 87 at 

 4    this point.  

 5               MR. SITCOV:  I will object.  It says draft on it.  

 6    It doesn't have a Bates stamp number.  

 7               THE COURT:  Isaki is who?  

 8               MR. SITCOV:  Isaki is an employee of the Census 

 9    Bureau.  

10               MR. SHERMAN:  In charge of the smoothing 

11    operation, as I understand it, one of the individuals in 

12    charge.  

13               MR. SITCOV:  I don't know if that's true, either, 

14    your Honor.  

15               THE COURT:  Pardon me?  

16               MR. SITCOV:  I say I don't know that that is 

17    true, that he is one of the individuals in charge of the 

18    smoothing operation.  It says a draft.  

19               MR. SHERMAN:  It is a draft, but, again, it goes 

20    to the proposition of alternatives of sigma squared I at the 

21    Census Bureau.  

22               THE COURT:  I will accept that on that basis.  87 

23    is admitted.

24               (Plaintiff's Exhibit 87 marked for identification 

25    was received in evidence.) 
                                                              2574

 1    BY MR. SHERMAN:

 2         Q.    I have handed you what is marked as PX 92, Dr. 

 3    Rolph.

 4               Can you describe to the court what this is?

 5         A.    Yes.  This is another memorandum dated December 

 6    4, 1990.  It's to Cary Isaki from Bill Bell entitled, "Error 

 7    model selection for regression smoothing," and as I 

 8    understand this document --  

 9               THE COURT:  Who is Bill Bell? 

10               THE WITNESS:  Bill Bell is a statistician who is 

11    employed by the Census Bureau and was involved in a number 

12    of these statistical analyses, as I understand it.  

13               I know he is employed by the Census Bureau, I 

14    understand from the document.

15         Q.    In your view, does this document represent the 

16    Census Bureau's test of the assumption represented by sigma 

17    squared I?

18         A.    Yes.  As I read the document, this is a case 

19    where they looked at several different alternatives to sigma 

20    squared I, and there is a discussion at the very end, I call 

21    your attention to the last, next to last sentence --  or the 

22    first sentence of the last paragraph, which says, "Based on 

23    your work to date, I would conclude that S equals V 3 and T 

24    equals sigma squared I are the best choice you got right 

25    now."
                                                              2575

 1         Q.    And what does that indicate to you?

 2         A.    That indicates to me that they have looked at 

 3    several alternatives to sigma squared I and after doing some 

 4    analyses have concluded that it is the best assumption in 

 5    terms of producing the best estimates.  

 6               MR. SHERMAN:  Your Honor, we would move PX 92.  

 7               MR. SITCOV:  I would object.  

 8               THE COURT:  92 is admitted.

 9               (Plaintiff's Exhibit 92 marked for identification 

10    was received in evidence.) 

11    BY MR. SHERMAN:

12         Q.    Dr. Rolph, did you hear Dr. Freedman's testimony 

13    about the work in his smoothing papers?

14         A.    Yes, I did.

15         Q.    Did it change your opinion about the results of 

16    the Bureau's smoothing operation?

17         A.    No, it did not.

18         Q.    Why not?

19         A.    Well, basically, three reasons, really:

20               The first one is that, as I listened to Dr. 

21    Freedman and read his papers, it was essentially an argument 

22    by analogy.  By that I mean he created his own models which 

23    represented some changes from the Bureau's models and then 

24    within those models tried a number of different things and 

25    then concluded because there were differences in those 
                                                              2576

 1    models those differences would carry over in the real world, 

 2    the Bureau real world, and just certainly based on his 

 3    testimony and reading his papers I don't find that analogy 

 4    sufficiently persuasive to persuade me that those 

 5    relationships hold in the real world.

 6               The second reason is that Dr. Freedman looked at 

 7    a number of different pieces, if you will, of the smoothing 

 8    and the presmoothing operation and asked the question what 

 9    would happen if you eliminated this piece or you eliminated 

10    these two pieces and made comparison across those, and that 

11    seemed to me to be rather like asking if you were driving a 

12    car down the road whether you took one of the spark plugs 

13    out or took a wheel off.  

14               I mean, basically, we are taking something that 

15    was designed to go together and taking something out of it 

16    and asking how well it worked, and that was not something I 

17    found terribly persuasive in terms of criticizing the 

18    procedures.

19               And the last, but not least, the comparisons he 

20    made were by and large comparisons of rather larger 

21    geographical areas, region would be an example, and the 

22    smoothing model was really designed to reduce sampling 

23    error, and in particular for smaller groups of the 

24    population, and specifically to the extent smoothing works 

25    or doesn't work, you got to look at smaller areas and if you 
                                                              2577

 1    look at these larger areas, particularly if you eliminate 

 2    the step that the Bureau took to try to make them agree, 

 3    ratio adjusting or benchmarking, it was not terribly 

 4    surprising to me that you would see some differences.

 5               So that I didn't find that to be in some sense 

 6    information that would change my mind.

 7         Q.    I would like to go back to the first of the 

 8    points that you mentioned, Dr. Freeman's changing of the 

 9    Bureau's model.

10               If we could, I would like to walk through the 

11    steps of the smoothing model.

12         A.    Okay.

13         Q.    Let's start with the functional form.  What was 

14    the Bureau's functional form for the smoothing model?

15         A.    Okay.  We are talking now sort of two steps, 

16    first looking at the variances and then the raw adjustment 

17    factors, so let's talk about the variances.

18         Q.    Very well.

19         A.    As I think was brought out in Dr. Freedman's 

20    testimony --  let me start with the Bureau and then talk 

21    about Dr. Freedman so we can keep things straight hopefully.

22               What the Bureau did was to use a functional form 

23    to model the variance which, let me use the shorthand, did 

24    not involve logarithms.

25               Dr. Freedman's model used, in one case, a model 
                                                              2578

 1    which involved logarithms, in another case a model which did 

 2    not involve logarithms, so that was one difference.

 3         Q.    What about the choosing of the variables for the 

 4    regression in the smoothing of variances?

 5         A.    Okay.

 6               The Bureau, as I understood it from the 

 7    documentation, developed a candidate set of variables for 

 8    the regression for modeling the variances and then selected 

 9    those that had T values greater than 2.

10         Q.    Do you remember what Dr. Freedman said about the 

11    Bureau's method?

12         A.    Memory isn't my strongest point, but my 

13    recollection was he said he had a conversation I think with 

14    Cary Isaki in which Dr. Isaki said something along the lines 

15    of --  I can't quote him, but there wasn't a systematic 

16    study devoted to selecting those factors.  I don't recall 

17    Dr. Freedman's words exactly.

18         Q.    And what did Dr. Freedman do in selecting 

19    variables in presmoothing?

20         A.    A couple of different things.

21               In one of his models he used the same variables 

22    that he had used in his regression model for the adjustment 

23    factors themselves, for others of his models he used a set 

24    of variables that the Bureau used.

25         Q.    So some of his models weren't the same as the 
                                                              2579

 1    Bureau's?

 2         A.    The same as the Bureau's model.

 3         Q.    What about in selecting outliers in modeling of 

 4    variances, what did the Bureau do?

 5         A.    What the Bureau did was to label as an outlier a 

 6    point that was more than four standard errors away from the 

 7    regression surface.

 8               What I understand Dr. Freedman to have done is in 

 9    some of his models not tried to identify outliers at all, in 

10    others as I understood him he used the same post-strata as 

11    the Bureau did.

12         Q.    What about in ratio adjustment of the variances?

13         A.    Again, what the Bureau did is they ratio adjusted 

14    the variances.  

15               I guess it's probably worth in hopes of the 

16    extent we can clarify all this, Dr. Freeman's term for ratio 

17    adjustment as indicated was benchmarking and I think in his 

18    first paper he called it benchmarking the variances, I think 

19    in the second paper, the thicker of the two, he calls it 

20    scaling the variances, if I'm recalling correctly.

21               The Bureau did ratio adjusted variances.  As I 

22    understand it, Dr. Freedman did not some of the time and did 

23    some of the time, so that would be the modeling of the 

24    variances in the presmoothing.

25         Q.    What about in estimating the covariance matrix?
                                                              2580

 1         A.    What the Bureau did to estimate the covariance 

 2    matrix was to model the variances or presmooth them as we 

 3    have discussed before and then estimate the covariance 

 4    matrix by multiplying those modeled variances times the 

 5    correlation matrix.

 6         Q.    What did Dr. Freedman do?

 7         A.    Dr. Freedman did that some of the time and didn't 

 8    do it some of the time.

 9         Q.    Dr. Rolph, moving on to smoothing the adjustment 

10    factors themselves, how did the Bureau choose the variables 

11    to be used in the regression equation?

12         A.    For the variables themselves, the Bureau 

13    developed two lists, basically.  

14               The first list consisted of a must variables, 

15    which were things like age, sex, race and so forth which 

16    were designed to preserve the demographic information in the 

17    adjustment factors.

18               The second list was what they called the 

19    candidate list, and this consisted of a list that came out 

20    of their several decades of experience in modeling the 

21    undercount, studying the undercount as well as the dress 

22    rehersal and TARO data.  They then used the best subsets 

23    technique to select from the candidate list to get a 

24    different equation for each of the four regions.

25         Q.    What is best subsets, Dr. Rolph?
                                                              2581

 1         A.    It's a method of choosing amongst --  choosing 

 2    from a group of variables to get one that fits the best as 

 3    defined in a suitable way.

 4         Q.    And what is Mallows C P?

 5         A.    Mallows C sub P is a particular method for 

 6    evaluating the fit of a regression equation.

 7         Q.    So is it fair to say it is part of the best 

 8    subsets regression?

 9         A.    My terminology would be it would be a method 

10    which you could use in conjunction to do a type of best 

11    subsets regression.

12         Q.    Did Dr. Freedman use the same technique with the 

13    Bureau in choosing his variables?

14         A.    As I understand it, no.  For some of his models 

15    he took the Bureau's list of candidate variables, part of 

16    that list, and then used a different method of selecting 

17    from it, the method being if the T ratio was greater than 2, 

18    I think he did a variant where the T ratio was greater than 

19    one a half if I recall correctly, for some of his models, 

20    for others --  I'm having a hard time keeping it straight --  

21    on others I think he did use the same set of variables, but 

22    I'm a little unclear on that.

23         Q.    In any case, did he use the same set of variables 

24    in the same selection technique?

25         A.    No.
                                                              2582

 1         Q.    With regard to the error structure used in 

 2    fittings the regression, what did the Bureau use?

 3         A.    The Bureau used what is called, pardon the term 

 4    of art, generalized least squareds with the relevant term 

 5    being our famous sigma squared I plus V 3.

 6               As I understood Dr. Freedman, he used --  and 

 7    that's the appropriate thing to use for the model the Bureau 

 8    fit.

 9               But what Dr. Freedman did, even with the same 

10    basic error structure, used a different generalized least 

11    squared formula which was not the appropriate one strictly 

12    speaking for that model.

13         Q.    What was the symbol for that model?

14         A.    I think it was the symbol K and what is missing 

15    is our friend sigma squared I.

16         Q.    In identifying outliers, Dr. Rolph, what did the 

17    Bureau do?

18         A.    The Bureau defined as an outlier in the 

19    adjustment regression points that were more than three 

20    standard errors away from the regression surface.

21         Q.    Is that what Dr. Freedman did?

22         A.    In some of his models, as I understood it, he did 

23    not do anything about outliers, in others he did.

24         Q.    What did the Bureau use as an estimation method?

25         A.    The Bureau used a procedure called maximum 
                                                              2583

 1    likelihood estimation, and Professor Freedman used for some 

 2    of his models, he used a method called the method of moments 

 3    and for others of his models he used a maximum likelihood 

 4    method, although I was not totally clear from the 

 5    documentation whether it was exactly the same one as the 

 6    Bureau.

 7         Q.    Is there a difference between those two?

 8         A.    Yes, I would describe method of moments, and I've 

 9    used it myself, not to knock it at all, as more of a quick 

10    and dirty method and maximum likelihood as something which 

11    you want to do when you want to all the bells and the 

12    whistles.

13         Q.    What about ratio adjustment of the adjustment 

14    factors?

15         A.    The Bureau did ratio adjustment of the adjustment 

16    factors.  Professor Freedman did it in some of his models 

17    and not in some of his other models.

18         Q.    Given all these differences, Dr. Rolph, what is 

19    your opinion about whether Dr. Freedman's work provides a 

20    valuable inference about the Bureau's smoothing process?  

21               MR. SITCOV:  I object.  

22               THE COURT:  Overruled.

23         A.    As I indicated earlier, I found the, if you will, 

24    analogy not close enough to really be persuasive to me in 

25    making a statement about what the Bureau did.  
                                                              2584

 1               I'm sure there are valid inferences about what 

 2    Professor Freedman did, but I didn't find the analogies 

 3    strong enough to draw conclusions about what the Bureau did, 

 4    that is, the real world, as it were.

 5         Q.    Dr. Rolph, you also mentioned I think it was 

 6    point two that you made Dr. Freedman's analysis of the 

 7    interconnecting parts that made up the Bureau's smoothing 

 8    model.

 9               Was this another term for testing the robustness 

10    of the model?

11         A.    Well, again, I believe in his testimony he termed 

12    it robustness.

13               My understanding of the term robustness is 

14    somewhat different than that.

15               What he did was to take particular pieces, if you 

16    will, of this set of steps that was going into producing 

17    these estimates and then removed them and then asked what 

18    difference it made.

19               What I would view robustness as being is the 

20    looking at departures from assumptions where you are looking 

21    at the whole procedure and seeing how well it works.

22               Just to use the car example, at the risk of 

23    overusing it, that would be the case where instead of 

24    pulling off a spark plug you would ask whether it operated 

25    well on wet streets, snowy streets, a hundred degree weather 
                                                              2585

 1    and so forth, really looking at variations at the whole 

 2    instead of taking the pieces apart.

 3         Q.    So in your view was Dr. Freedman's analysis a 

 4    proper robustness analysis?

 5         A.    Well, I certainly didn't find it persuasive as an 

 6    exercise in testing whether the model for robust.

 7         Q.    Dr. Rolph, did the Bureau do any tests of the 

 8    robustness of the smoothing model?

 9         A.    Yes, they did, as I understand it.

10         Q.    I have handed you PX 118.

11               Can you tell the court what this is?

12         A.    Yes.  It's headed at the top, "Tom Belin, May 21, 

13    1991, tech design meeting notes."

14         Q.    What does this represent?

15         A.    As I read it, it is looking at several 

16    alternatives --  

17               THE COURT:  Who is Tom Belin?  

18               MR. SHERMAN:  I'm not sure if he still is, your 

19    Honor. 

20               THE WITNESS:  If I might, he was employed by the 

21    Bureau, he was a graduate student I think at the time, he is 

22    now on the staff at UCLA, actually.

23         Q.    As I understand it from reading this memo, it is 

24    Tom looking at a number of different alternative models in 

25    an effort to see how sensitive the results are to slightly 
                                                              2586

 1    different specifications.

 2         Q.    Would you call this a test of the robustness of 

 3    the smoothing procedure?

 4         A.    I would call this a sensitivity analysis, and 

 5    that's the term I would use to test robustness.  

 6               MR. SHERMAN:  We move Plaintiff's Exhibit 118, 

 7    your Honor.  

 8               MR. SITCOV:  No objection.  

 9               THE COURT:  118 is admitted.

10               (Plaintiff's Exhibit 118 marked for 

11    identification was received in evidence.) 

12    BY MR. SHERMAN:

13         Q.    I have handed you PX 126.

14               Can you tell the court what this is?

15         A.    Yes.  It's headed, "June 6, 1991, William Bell 

16    statistical research division." 

17               I mentioned Dr. Bell is a statistician at the 

18    Bureau.  

19               It's headed, "Raw and smoothed, from various 

20    models, adjustment factors and census adjustments, tables 

21    and graphs," and what it describes is some analysis which 

22    looks at two alternative methods of smoothing labeled Fay 1 

23    and Fay 2, which are identified as Bob Fay's model, and 

24    presents results from those analyses.

25               So, again, I would call this a sensitivity 
                                                              2587

 1    analysis, an effort to make sure that what they were using 

 2    was reasonably robust.  

 3               MR. SHERMAN:  We move 126, your Honor.  

 4               MR. SITCOV:  No objection.  

 5               THE COURT:  126 is admitted.

 6               (Plaintiff's Exhibit 126 marked for 

 7    identification was received in evidence.) 

 8    BY MR. SHERMAN:

 9         Q.    I have handed you PX 127. 

10               Will you describe to the court what that is?

11         A.    Yes.  This is headed, "June 10, 1991, William 

12    Bell, statistical research division," it is headed, 

13    "Addition to handout to June 6, tilted," and it gives the 

14    title as I gave to the previous document.

15               What it appears to be in its title and what is 

16    tabulated on the page is some additional material relating 

17    to the analysis on the previous document, again, an example 

18    of, as I read it, sensitivity analysis of the model of 

19    robustness.  

20               MR. SHERMAN:  We move 127.  

21               MR. SITCOV:  No objection.  

22               THE COURT:  127 is admitted.

23               (Plaintiff's Exhibit 127 marked for 

24    identification was received in evidence.) 

25    BY MR. SHERMAN:
                                                              2588

 1         Q.    I have given you PX 129, Dr. Rolph.

 2               Will you describe to the court what that is?

 3         A.    It's headed, "Census raw adjusted, modified 

 4    smoothed, smoothed counts," and at the top it says 10:44, 

 5    Tuesday, June, on my copy a blank, 1991.

 6         Q.    And what is it?

 7         A.    It appears to be a series of tables which 

 8    indicate the results of running or fitting different models 

 9    to the data, so it appear to be another example of looking 

10    at the sensitivity of what your assumptions are.

11         Q.    And, therefore, a test of the robustness?

12         A.    Yes.  

13               MR. SHERMAN:  We move 129.  

14               MR. SITCOV:  I object and I will move to strike.

15               It starts out on page 6, your Honor.  I don't 

16    know what it is page 6 of.  The witness says he only things 

17    it appears to be something.

18               I certainly thing we are entitled to more than 

19    that before this is admitted into evidence.  

20               THE COURT:  I also note the last two pages are 

21    numbered 6 and 7 as well. 

22               THE WITNESS:  I thought that was only my copy.  

23               MR. SHERMAN:  I have no explanation, your Honor.  

24               THE COURT:  I will sustain your objection, Mr. 

25    Sitcov, and strike the last statement of the testimony.  
                                                              2589

 1               MR. SHERMAN:  My next one starts with page 2, so 

 2    I guess I will forego that.  

 3    BY MR. SHERMAN:

 4         Q.    Dr. Rolph, the documents that you have just gone 

 5    through with the court and me, are they all dated prior to 

 6    the Secretary's decision?  

 7               THE COURT:  You can tell us that, can't you?  

 8               MR. SHERMAN:  They are.

 9         A.    They are, I agree.

10         Q.    Are you aware of any other sensitivity analysis 

11    or tests for robustness done on the Bureau's smoothing model 

12    prior to the Secretary's decision?

13         A.    Yes.  There were a number that were done which 

14    are documented in the work that was work that was done by, 

15    for example, the technical advisory committees work.

16               I have in mind --  what comes immediately to mind 

17    is a study by Hoaglin and Glickman that looked at a number 

18    of regression models --  

19               MR. SITCOV:  Your Honor, I am going to object.  

20    This is all hearsay.  If there is such a study, let's see 

21    it.  But I certainly object to his testimony about other 

22    studies without us being able to even look at them.  

23               THE COURT:  How do you know there are these other 

24    studies? 

25               THE WITNESS:  I read the study that I am 
                                                              2590

 1    referring to right now.  It's part of the report of Estrada, 

 2    Ericksen, Wolter and Tukey.  

 3               THE COURT:  I will permit the testimony to stand 

 4    that he knows these things exist, that he read them, without 

 5    going into their context.  

 6               MR. ZIMROTH:  It's also already in evidence, your 

 7    Honor.  

 8               THE COURT:  Okay.

 9         Q.    Are you talking about the Hoaglin and Glickman 

10    report?

11         A.    Yes, I am.  

12               MR. SHERMAN:  For the record, your Honor, that is 

13    appendix E to PX 195.           

14    BY MR. SHERMAN:

15         Q.    Will you describe to the court what that study 

16    is?

17         A.    Yes.  That was exploring a number of different 

18    regression models for the smoothing operation -- if I recall 

19    I think there were 13 different ones --  that they looked at 

20    and then looked at what the results were of using these 

21    different models.  

22               The tables that come to mind from that report of 

23    ones that look at various evaluation of post-strata and 

24    their overall conclusion was that they got approximately the 

25    same result no matter which model they used.
                                                              2591

 1         Q.    In your view, Dr. Rolph --  let me ask you a 

 2    question before that.

 3               Did the Hoaglin and Glickman study and the 

 4    various Bureau documents that we looked at do analysis of 

 5    the actual Bureau data?

 6         A.    Yes, that was analysis of Census Bureau data from 

 7    the 1990 census and PES.

 8         Q.    And in your view, were these proper tests of the 

 9    robustness of the smoothing model?

10         A.    Yes.  I thought they were carried out very well.  

11    I have the highest regard for the statisticians at the 

12    Bureau and I thought they did an excellent job.

13         Q.    Dr. Rolph, what was the purpose of smoothing the 

14    adjustment factors?

15         A.    To reduce the sampling error of the raw 

16    adjustment factors from the 1,392 post-strata.

17         Q.    Do you have a view about whether that was 

18    successfully done?

19         A.    Yes.  I believe in my direct testimony I even 

20    referred to a chart, I think, which showed how much the 

21    sampling error was reduced in various subgroups of the 

22    population.  

23               MR. SHERMAN:  If I might, your Honor, can we 

24    visit that chart just briefly?  

25               THE COURT:  All right.  
                                                              2592

 1               MR. SHERMAN:  This is marked as PX 722, it has 

 2    been admitted.  

 3    BY MR. SHERMAN:

 4         Q.    Is that the chart you were referring to, Dr. 

 5    Rolph?

 6         A.    Yes.  What that chart shows is for the different 

 7    racial groups, subgroups of the population, the difference 

 8    between the estimated standard error used in the smoothed 

 9    versus the unsmoothed models and what it shows is for the 

10    racial minorities, you tend to get a considerable 

11    improvement in reducing the sampling error in using 

12    smoothing as opposed to not using smoothing.

13         Q.    Dr. Rolph, did the testimony that you heard from 

14    Dr. Freedman cause you to change your opinion about whether 

15    smoothing was properly done by the Bureau?

16         A.    No, it did not.  

17               MR. SHERMAN:  I have nothing else, your Honor.  

18               THE COURT:  Thank you, Mr. Sherman.

19               Any cross?  

20               MR. SITCOV:  Yes.

21               Could we have a short break, your Honor?  

22               THE COURT:  How long do you want?  

23               MR. SITCOV:  Half an hour?  Will that be all 

24    right?  

25               THE COURT:  Surely you jest.  
                                                              2593

 1               MR. SITCOV:  Twenty minutes?  

 2               THE COURT:  All right.

 3               Is there anybody else today?  

 4               MR. ZIMROTH:  Nobody else today.           

 5               MR. ZIMROTH:  I have an application that I wanted 

 6    to make, but we will wait.  

 7               THE COURT:  Until quarter of four?  

 8               MR. ZIMROTH:  No, until after the examination.

 9               (Recess) 

10    

11    

12               (Continued on the next page.)

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2594

 1               MR. ZIMROTH:  Your Honor, before we begin, may I 

 2    make an application?  

 3               THE COURT:  Sure.  

 4               MR. ZIMROTH:  Your Honor, at this time the 
                                                            

 5    plaintiffs would like to renew their application to depose 

 6    and call as a trial witness former Secretary Robert 

 7    Mosbacher.  Throughout this trial the defendants have hidden 

 8    behind his absence.  When asked about a particular document 

 9    or page in his report, the witnesses have been cross- 

10    examined on the grounds that it is pure speculation, they 

11    don't know what was in the Secretary's mind when he wrote 

12    whatever he wrote or whether he considered whatever he 

13    considered.  

14               In addition, they have shielded their own 

15    witnesses from damaging documents and evidence on the 

16    grounds that when we try to put the evidence in, we could 

17    not prove what the Secretary actually saw or considered.  I 

18    give you just one example now.  This is in Mr. Bounpane's 

19    redirect testimony.  In his cross-examination, Mr. Bounpane 

20    was asked about the ethnographic studies, and he said that 

21    he would be surprised if the Secretary didn't consider them.  

22               On redirect, I believe it was Mr. Subar:

23              "Q.    Mr. Cone asked you," he us talking to 

24    Bounpane now, "some questions about ethnographic studies.  

25    Do you know whether the Secretary of Commerce had the 
                                                              2595

 1    ethnographic studies available to him before he made his 

 2    decision?

 3              "A.    No, I don't.

 4              "Q.    Do you know whether Barbara Bryant had them 

 5    available to her before she made her recommendation?

 6              "A.    No, I don't."  

 7               Mr. Mosbacher's absence again was used as both a 

 8    shield and a sword here, which is exactly what the 

 9    defendants are doing.  For that reason, we renew our 

10    application to take his deposition and to call him as a 

11    witness.  

12               THE COURT:  Thank you, Mr. Zimroth.  

13               Mr. Millet?  

14               MR. MILLET:  Your Honor, there is nothing new 

15    that has been proffered in this trial to prompt the Court to 

16    reconsider the rulings that you have already issued in this 

17    case.  This is not rebuttal testimony that they are 

18    offering.  There is not anything that they are offering by 

19    way of rebutting the direct testimony that we offered.  That 

20    is the point of this stage of the proceedings.  

21               We are not using the former Secretary as a shield 

22    or a sword in any manner.  The only thing that Mr. Zimroth 

23    has referred to that is in any way different are the 

24    ethnographic studies.  Your Honor, as we argued when these 

25    studies were first attempted to be admitted and your Honor 
                                                              2596

 1    admitted them conditionally, on the face of most of those 

 2    studies, they post-dated the Secretary's decision.  That was 

 3    the basis for our objection to those, and you acknowledged 

 4    that that was the reason for our objecting to them.  Your 

 5    Honor allowed them to be admitted subject to plaintiffs 

 6    connecting them up at some point later in this proceeding.  

 7               That by itself, and it is the only thing I hear 

 8    Mr. Zimroth refer to, provides no reason to reconsider the 

 9    Court's earlier ruling on why the Secretary should or should 

10    not be called.  For that matter, he is the former Secretary, 

11    and I'm not sure that we could find him at this point.  He 

12    doesn't work for us.  

13               MR. ZIMROTH:  Your Honor, I did just read one 

14    example, but there are many others in the testimony.  

15    Professor Cain on cross-examination was asked, "You don't 

16    know, do you, that Secretary Mosbacher considered the 

17    political implications of adjustment as an actual fact?" 

18               The Court says, "It's an inference on your part?

19              "A.    It's an inference based on the fact that 

20    these things are obvious, that is correct.

21              "Q.    For all you know, he made efforts to 

22    isolate himself from political forces, is that correct?

23              "A.    I don't know.  Once you know something, I 

24    don't know how you unknow it.  He may have tried to unknow 

25    things."  
                                                              2597

 1               The point being that they have been cross- 

 2    examining our witnesses again and again on the grounds that 

 3    our witnesses are in no position to know what Mr. Mosbacher 

 4    took into account and what he didn't take into account, what 

 5    influences were brought to bear and what not.  I think that 

 6    that is grounds for our having a chance andm franklym for 

 7    the Court having a chance to hear Mr. Mosbacher.  

 8               THE COURT:  I am not satisfied that the 

 9    plaintiffs have made the factual showing required by United 

10    States v. Morgan.  Therefore, the application to subpoena 

11    Mr. Mosbacher is denied.  

12               Cross-examination, Mr. Sitcov.  

13               MR. SITCOV:  Thank you, your Honor.  I apologize 

14    in advance if it is a little disjointed.  

15               THE COURT:  I won't be able to tell.  

16               MR. SITCOV:  Thank God this is not televised.  

17    CROSS-EXAMINATION 
                        

18    BY MR. SITCOV:

19         Q.    Dr. Rolph, you are aware, aren't you, that Dr. 

20    Freedman replicated the Bureau's smoothing model?

21         A.    Exactly?

22         Q.    You heard him testify that he replicated the 

23    Bureau's smoothing model, didn't you?

24         A.    Quite honestly, I don't recall whether his words 

25    said hta he exactly replicated.  I know he replicated 
                                                              2598

 1    something he thought was very close to it.

 2         Q.    You didn't replicate the Bureau's smoothing 

 3    model, did you?

 4         A.    I certainly did not.  

 5               MR. SITCOV:  May I enter the well for a second, 

 6    your Honor?  

 7               THE COURT:  Sure.  

 8               MR. SITCOV:  You'll love this, your Honor, trust 

 9    me.

10         Q.    I am going to put it over here.  Can you see 

11    this?

12         A.    Yes, I can see it.

13         Q.    Do you see the second equation there, that says Y 

14    equals X beta plus V?

15         A.    Second from the bottom there or the top?  Where 

16    are we?

17         Q.    I'm sorry.  Third line from the bottom, Y equals 

18    X beta plus V?

19         A.    Yes, I do see that.

20         Q.    The sampling covariance matrix is sigma, is that 

21    right?

22         A.    Not on that sheet.  The sample covariance matrix 

23    is V 3, if I understand that sheet.

24         Q.    Let's do it hypothetically that Y equals X beta 

25    plus B.  In the sampling, the covariance matrix for V is 
                                                              2599

 1    sigma?

 2         A.    Okay.

 3         Q.    Then a GSL estimate for beta should be based on 

 4    sigma, is that right?

 5         A.    If that is all you are assuming, yes.

 6         Q.    You testified about the Hoaglin & Glickman report 

 7    on your direct examination.  Do you recall that?

 8         A.    Yes, I do.

 9         Q.    Do you recall that when you were deposed last 

10    month, you testified that you didn't fully understand 

11    certain things about the Hoaglin and Glickman report?

12         A.    I recall when I was deposed I was shown several 

13    pages which at the time I don't believe I knew were from the 

14    Hoaglin and Glickman report by memory, and I was not able to 

15    decipher them for Mr. Baron, is my best memory.

16         Q.    You recall saying that you don't fully understand 

17    how the weighting is being plotted?

18         A.    I suspect I said even vaguer things than that.

19         Q.    Now, it is correct, isn't it, that you hadn't 

20    performed any independent analyses of the 1990 production 

21    smoothing model, is that right?

22         A.    Yes.  I still have not.

23         Q.    You haven't performed any independent 

24    calculations regarding how much uncertainty there is in the 

25    variance of the smoothed adjustment factors, isn't that 
                                                              2600

 1    right?

 2         A.    That is also correct, yes.

 3         Q.    You haven't performed any statistical analyses on 

 4    any census-related adjustment issue since the time you have 

 5    been retained as a consultant in this case, isn't that 

 6    right?

 7         A.    That's correct, yes.

 8         Q.    It would be correct to describe the presmoothing 

 9    process as a situation where variances were estimated from 

10    survey data by the jackknife and then were presmoothed by 

11    regression, is that right?

12         A.    That sounds about right to me.

13         Q.    Do you recall testifying on May 15th in this case 

14    that you couldn't think of any other example besides the 

15    1990 proposed adjustment of where variances had been 

16    estimated from survey data by the jackknife and then the 

17    estimates were presmoothed by a regression?

18         A.    I don't recall that precisely.  But if you 

19    represent that, I have no reason to disbelieve you.

20         Q.    That is a true statement, isn't it?

21         A.    I'm saying I don't disbelieve you.  I can't 

22    recall my exact answer.  That sounds right.  

23               THE COURT:  Supposing he put the question to you 

24    now afresh.

25         Q.    You are not aware of any other examples where 
                                                              2601

 1    variances have been extimated from survey data by the 

 2    jackknife and then the estimates were smoothed by 

 3    regression?

 4         A.    Actually, if you look at some of the CPS 

 5    documents that I was asked about in my rebuttal testimony, 

 6    they do something very similar to that but not exactly that. 

 7         Q.    Do you have Plaintiffs' Exhibit 763 in front of 

 8    you?

 9         A.    Yes, I do.

10         Q.    The model variances in Plaintiffs' Exhibit 763 

11    are not used as inputs to a smoothing model, are they?

12         A.    Not to my knowledge.

13         Q.    Do you have Plaintiffs' Exhibit 70?  Do you have 

14    that in front of you?

15         A.    Seven zero?

16         Q.    Yes.

17         A.    Just a moment.  Yes, I do.

18         Q.    According to the third sentence in the first 

19    paragraph, Bill Bell preferred using a simple model term 

20    sigma squared I and the smoothed variances term V sub 3, is 

21    that correct?

22         A.    I think you have read that correctly, yes.

23         Q.    It doesn't say why he preferred that, does it?

24         A.    No, it does not say that here.

25         Q.    Do you have Plaintiffs' Exhibit 87 in front of 
                                                              2602

 1    you?

 2         A.    Yes, I do.

 3         Q.    Exhibit 87 is based on 1988 data, is it not?

 4         A.    Yes, it is.

 5         Q.    It doesn't even refer to 1990 data, does it?

 6         A.    That's correct.

 7         Q.    In fact, on the second page, the third line from 

 8    the bottom, the whatever you want to call this, memorandum, 

 9    notes that it prefers sigma squared I to sigma squared V 

10    because the former appears to give smaller estimated 

11    standard errors, isn't that right?

12         A.    It reads actually smaller standard errors, if 

13    that is what you are referring to, yes.

14         Q.    Yes.

15         A.    Sigma squared V zero, I think.

16         Q.    Yes.

17         A.    Yes.

18         Q.    Do you have Plaintiffs' Exhibit 92 in front of 

19    you?

20         A.    Yes, I do.

21         Q.    Can you look at the second line in the first 

22    paragraph.  It says, "I computed AIC values."

23         A.    Yes, I do.

24         Q.    What does AIC stand for?

25         A.    I would infer it means -- you are not going to 
                                                              2603

 1    like this -- Akaike's information criterion, I believe. 

 2         Q.    Are you aware of any other work which uses 

 3    Akaike's criterion to compare nonnested models?

 4         A.    I guess I would have to say that I wouldn't know 

 5    one way or the other.  I come across it being used, and the 

 6    answer is no, I guess.

 7         Q.    Do you see in the first line in the first 

 8    paragraph the reference to 2 log L?

 9         A.    Yes, I do.

10         Q.    Do you know what L is?

11         A.    I would infer from its use here that it is what 

12    is called the likelihood function.

13         Q.    Are you aware of any other work which uses 

14    negative 2 log L to compare nonnested models?

15         A.    I certainly am aware of using minus 2 log L 

16    routinely.  I would have a hard time saying whether they are 

17    nested or nonnested without going back and looking at some 

18    paper.  But it is a quite standard thing to calculate.

19         Q.    As you sit here today, you don't know of any 

20    other work which uses their nonnested models?

21         A.    Not one way or the other, no.

22         Q.    The comparison in Plaintiffs' Exhibit 92 is of 

23    two nonnested models using 1988 data, is that right?

24         A.    As I read it, it refers to, I think, about four 

25    models, if I am not mistaken.  It talks about model 1, model 
                                                              2604

 1    2, model 4.  I am somehow not focusing on model 3.  So I 

 2    think it is more than two models as I read it.  I'm sorry, I 

 3    lost the other part of your question.  Oh, nonnested?

 4         Q.    Yes.

 5         A.    I am not sure from looking at this, but it could 

 6    be that the model which says T equals sigma squared C and T 

 7    equals sigma squared I would be nested.  But it would depend 

 8    on some details which I am just not clear on from looking at 

 9    this memo.  

10               So the answer is I'm not certain about the answer 

11    to your question.

12         Q.    Can you look at, I think it is, the third 

13    sentence in the first paragraph, the one that says, "It 

14    seems to me comparisons are only valid between models with 

15    the same sampling covariance (S) and with regression 

16    variables (M) determined the same way.

17         A.    I see that same sentence.

18         Q.    Could you look down into the last paragraph?

19         A.    Okay.

20         Q.    It says, "Based on your work to date, I would 

21    conclude S equals V sub 3 and T equals sigma squared I are 

22    the best choice you have got right now."  

23               Do you see that?

24         A.    Yes, I do.

25         Q.    Those are two different covariance matrices, 
                                                              2605

 1    aren't they?

 2         A.    Yes, they are. 

 3         Q.    Do you recall being asked at your deposition last 

 4    month the following questions and giving the following 

 5    answers?  It is on page 914.

 6              "Q.    Let me switch gears for a moment.  Do you 

 7    know what folded wings are?  Have you ever heard of that 

 8    term?

 9              "A.    Only when making paper airplanes.

10              "Q.    Not in statistical analysis?

11              "A.    It doesn't come immediately to mind."  

12               Do you recall that?

13         A.    Yes, I do recall that particular question and 

14    that answer.

15         Q.    Do you have Plaintiffs' Exhibit 195 in front of 

16    you, that is, the report from Hoaglin and Glickman that I 

17    think you testified about earlier?

18         A.    I don't, but if you would put it in front of me, 

19    I would be happy to look at it.

20         Q.    Would you turn to page 26.

21         A.    Okay.

22         Q.    There is a section there entitled "Fitting 

23    Regressions to Adjustment Factors."  At the bottom of that 

24    first paragraph there it says, "Tukey developed these 

25    alternative treatments of covariances as a companion to the 
                                                              2606

 1    prescription for dealing with the variances.  The three are 

 2    as follows:  Use raw correlations, use zero correlations, 

 3    use folded wings."  

 4               Do you see that?

 5         A.    I see that, yes.

 6         Q.    Could you turn over to page 31, to the section 

 7    number 6, "Comparing Fits."  Paragraph number 2 says, "The 

 8    four combinations from 1" --

 9         A.    Pardon me.  Page 31?

10         Q.    31.  There is a section entitled "6.  Comparing 

11    Fits."

12         A.    Yes.

13         Q.    Then there are some numbered paragraphs.  Do you 

14    see that?

15         A.    Yes.  Okay.

16         Q.    The second paragraph, the one numbered 2, says, 

17    "The four combinations from one of (Point 6, Point 10) with 

18    one of (raw correlations folded wings)."  Do you see that?

19         A.    I see that, yes.

20         Q.    Are you being compensated for your work for 

21    plaintiffs in this case?

22         A.    I have been in the past, yes.

23         Q.    What is your rate of compensation?

24         A.    It was $200 an hour. 

25         Q.    Is it the same rate for testimony in depositions?
                                                              2607

 1         A.    No.  It is time and a half.

 2         Q.    So that is $300 an hour?

 3         A.    That's correct.  

 4               MR. SITCOV:  No further questions.  

 5               THE COURT:  Any redirect?  

 6               MR. SHERMAN:  No, I have nothing, your Honor.  

 7               THE COURT:  Thank you very much, Dr. Rolph.  It 

 8    is a pleasure to hear from you again.  

 9               (Witness excused) 

10               MR. MILLET:  Your Honor, on the chance that this 

11    may be the last day of trial, would it be appropriate to 

12    talk about post-trial briefing 

13               THE COURT:  All that needs to be done is work out 

14    a schedule, and I would ask you to meet with my clerk to 

15    work that out. 

16               MR. SOLOMON:  I apologize for this.  I should not 

17    have to raise this.  We asked your Honor after Dr. 

18    Freedman's cross for what he had that underlay his 

19    assertions concerning Professor Fisher.  

20               THE COURT:  Right.  Mr. Sitcov said he would get 

21    it for you.  

22               MR. SOLOMON:  Right.  After lunch I was given a 

23    disk which I think I should have been given weeks ago --but 

24    put that aside -- which does have some data on it, and then 

25    a sheet of paper, the sheet of paper purportedly being from 
                                                              2608

 1    what Dr. Fay did and the disk being some data.  

 2               We don't have the program.  We don't have the 

 3    methodology that was done.  This game should not be played.  

 4    We would request all of the data and the program so that we 

 5    can see what Dr. Freedman did.  We request that your Honor 

 6    give us leave to take the deposition of this person this 

 7    evening.  

 8               THE COURT:  What person?  

 9               MR. SOLOMON:  Of Dr. Freedman, to find out what, 

10    in fact, is going on.  I am happy to take Mr. Sitcov's 

11    representation that we have it if he will give it to us.  

12    Otherwise, especially if we are thinking about calling 

13    Professor Fisher, let's get the stuff so we can look at it.  

14               MR. SITCOV:  They had that, your Honor.  They 

15    have all of his data.  

16               THE COURT:  And the program that he says he 

17    needs?  

18               MR. SITCOV:  They do not have the program.  That 

19    is not the data he used.  The program is something that he 

20    developed.  They are entitled, under Rule 705 to the facts.  

21               THE COURT:  Can they interpret the data without 

22    the program?  

23               MR. SITCOV:  Absolutely.  They most certainly 

24    can.  

25               THE COURT:  Then why do you need the program, Mr. 
                                                              2609

 1    Solomon?  

 2               MR. SOLOMON:  First of all, I know of no rule 

 3    that limits Rule 705, the reference in 705, to the data and 

 4    not the programs.  In fact, when I read to your Honor 

 5    yesterday from Judge Weinstein is exactly to the contrary.  

 6    Second, I took this disk and I gave it to the computer 

 7    people, and they came back and they said, what is it you 

 8    want us to do with this because we don't have the 

 9    methodologies and we don't have the program, and how can we 

10    do it?  

11               The witness told me yesterday morning, you will 

12    remember, that without the program, if we have the data, 

13    that what we can do is we can try to recreate what he did 

14    and it will take between, in his view, two weeks and two 

15    months.  There is no reason for this.  

16               MR. SITCOV:  Rule 705 says, your Honor, "The 

17    expert may in any event be required to disclose the 

18    underlying facts or data."  He has disclosed all of his 

19    facts and data.  

20               THE COURT:  I don't have a clear picture in my 

21    own head as to what the difference is between a program and 

22    data.  

23               MR. SITCOV:  The data are the computer bits, that 

24    is the information that is on the computer disk.  The 

25    program would be the original work that Dr. Freedman did to 
                                                              2610

 1    utilize that.  

 2               The plaintiffs ought to have programs that do 

 3    these things because this work essentially recreates Dr. 

 4    Fisher's stuff.  

 5               THE COURT:  Why are you reluctant to give them 

 6    the program?  I don't have a feel for that either.  

 7               MR. SITCOV:  Mr. Solomon is acting as though we 

 8    are obliged by the rule to do so.  

 9               THE COURT:  Assuming you are not, would you do it 

10    as a favor for me?  

11               MR. SITCOV:  If we weren't obliged and we could 

12    do it as a favor for you, I would be happy to do it if it is 

13    possible to do easily.  I don't know if it is.  I suppose I 

14    would have to ask Dr. Freedman.  

15               THE COURT:  Do you want to do that?  Do you want 

16    to caucus?  

17               MR. SITCOV:  Yes.  Do you mind?  

18               THE COURT:  Not at all

19               (Pause) 

20               MR. SITCOV:  Your Honor, I am told by Dr. 

21    Freedman that part of it still exists in some sort of hard 

22    form, part does not.  He could recreate that if your Honor 

23    desires, and we could provide it to the plaintiffs.  I 

24    should note that according to Dr. Freedman, it would not 

25    take any near weeks or days, but a short period of time.  
                                                              2611

 1               On the other hand, we have not had any access to 

 2    anything of Dr. Fisher's.  We would certainly like to have 

 3    the programs that were used to create Dr. Fisher's charts, 

 4    and we would like them pretty quick.  

 5               THE COURT:  Ly.  

 6               MR. SITCOV:  Ly.  We would like them fastly.  

 7               THE COURT:  Sounds fair.  Mr. Solomon?  

 8               MR. SOLOMON:  The question of quickly is 

 9    completely unfair.  This is the first we heard of this 

10    request.  As to whether to give it to them, we offered it to 

11    them on May 1st.  Of course there is no problem with that.  

12    The facts, including the data and the programs, should be 

13    made available upon request.  We made that offer on May 1st.  

14    I have no problem with that.  

15               MR. SITCOV:  We asked for them on Sunday.  

16               MR. SOLOMON:  What Sunday?  

17               MR. SITCOV:  This past Sunday.  

18               MR. SOLOMON:  Sorry.  The request was not made of 

19    me.  If there is a problem, I see nobody coming up.  There 

20    is no problem.  

21               THE COURT:  In any case, you will do it, and work 

22    out a schedule between you.  

23               MR. SITCOV:  Yes.  

24               THE COURT:  I appreciate your cooperativeness.  

25               MR. RIFKIND:  While we are making mutual 
                                                              2612

 1    requests, there is a disagreement between me and Mr. Sitcov 

 2    this morning.  He made some representations about Dr. 

 3    Fisher's deposition with which I took exception.  The 

 4    deposition wasn't present.  He said it was in the next room.  

 5    I wish in the course of the evening he will let me know what 

 6    pages he was referring to in the Fisher deposition, because 

 7    I can't find them.  

 8               THE COURT:  You will do that?  

 9               MR. SITCOV:  Yes.  

10               THE COURT:  On the chance, paren read hope, that 

11    there is no further rebuttal -- it would be tomorrow 

12    afternoon -- I would like before we adjourn to commend 

13    counsel.  This case had the potential to be highly 

14    acrimonious.  I think it worked out rather well.  

15               It is a tribute to the professionalism and the 

16    civility of counsel for both sides that they were able to 

17    resolve their many procedural and substantive differences 

18    with the least bit of conflict.  I have rarely seen a case 

19    tried as well as this.  Indeed, it may well have been the 

20    best tried case I have ever presided over.  I congratulate 

21    counsel.  

22               I would also like to just mention to the 

23    statisticians here, I don't know whether you all know it or 

24    not, but I have spent almost all of my professional life in 

25    the groves of academe, including ten years as the dean of a 
                                                              2613

 1    law faculty.  So I think I know good teaching when I hear 

 2    it.  

 3               I expected to come out of this trial as ignorant 

 4    as when I entered it.  It may well be that I still am, but I 

 5    think I learned a great deal, and it was certainly a most 

 6    rewarding intellectual experience.  I can understand now why 

 7    the faculty at Berkeley is so highly regarded.  You may be 

 8    pleased to know that my next law clerk who will join me in 

 9    September is from the law school at Berkeley.  A Jesuit, by 

10    the way.  

11               It really has been a pleasure meeting all of you.  

12    If I don't get to see you again, I hope that I do have the 

13    pleasure somewhere, sometime to deal with all of you again.  

14    Thank you for an extremely well tried case.  

15               (Adjourned) 

16    

17

18

19

20

21

22

23

24

25
