                                                              2037

 1    UNITED STATES DISTRICT COURT
      EASTERN DISTRICT OF NEW YORK
 2    ------------------------------x
      THE CITY OF NEW YORK, et al.,
 3    
                         Plaintiffs,          88 Civ. 3474 (JMcL)
 4    
                 v.                           
 5    
      UNITED STATES DEPARTMENT OF COMMERCE,
 6    et al., 
      
 7                       Defendants.
      ---------------------------------x
 8    CITY OF ATLANTA, et al.,
                     
 9                       Plaintiffs,
      
10                v.                          92 Civ. 1566 (JMcL)
                                              
11    MOSBACHER, et al.,
      
12                       Defendants.
      ---------------------------------x
13    FLORIDA HOUSE OF REPRESENTATIVES, 
      et al.
14    
                         Plaintiffs,
15                v.                          92 Civ. 2037 (JMcL)
      
16    BARBARA H. FRANKLIN, Secretary of
      Commerce,
17    
                         Defendant.
18    --------------------------------x
      
19    
                                              May 26, 1992
20                                            9:30 a.m.
      
21    
      
22    Before:
      
23               HON. JOSEPH M. McLAUGHLIN,
      
24                                            Circuit Judge
      
25
                                                              2038

 1               THE COURT:  Any housekeeping matters?  

 2               MR. MILLET:  Not from our side, your Honor.  

 3               MR. ZIMROTH:  No, your Honor.  

 4               THE COURT:  All right.  Mr. Millet, do you have a 

 5    witness? 

 6               MR. MILLET:  We do indeed, your Honor.

 7               We are going to try to keep them short and sweet.  

 8               THE COURT:  Good.  

 9               MR. MILLET:  The defendants are going to call 

10    Professor Leo Breiman.

11    LEO BREIMAN,

12             called as a witness by the defendants, having

13             first been duly sworn, was examined and

14             testified as follows:) 

15               MR. MILLET:  Your Honor will find a copy of 

16    Professor Breiman's CV under tab five.

17    DIRECT EXAMINANTION

18    BY MR. MILLET: 

19         Q.    Professor Breiman, where are you employed?
                                                           

20         A.    University of California at Berkeley.

21         Q.    What is your position at the University of 

22    California at Berkeley?

23         A.    Professor of statistics.

24         Q.    How long have you been a professor of statistics 

25    at Berkeley?
                                                              2039

 1         A.    Twelve years.

 2         Q.    Could you describe for the court your educational 

 3    background?

 4         A.    I got my bachelor's degree at Cal Tech, my 

 5    master's degree in mathematics at Columbia University, my 

 6    Ph.D. at University of California at Berkeley in 

 7    mathematics.

 8         Q.    Following the completion of your post-graduate 

 9    studies, where were you employed?

10         A.    For two years an assistant professor at Berkeley 

11    and then from 1960 to 67 as a professor on the faculty of 

12    UCLA.  Following that I resigned from UCLA and went into 

13    freelance consulting for 13 years.

14         Q.    What was the department in which you were 

15    associated with at UCLA?

16         A.    It was the mathematics department, because at 

17    that point statistics was included in the mathematics 

18    department.

19         Q.    What were your subject matters of instruction at 

20    UCLA?

21         A.    Probability and statistics.

22         Q.    At the time you left UCLA, did you have tenure?

23         A.    Yes, I did.

24         Q.    Why did you decide to give up the Holy Grail and 

25    go out and become a consultant?
                                                              2040

 1         A.    Well, my basic training was as a theoretician in 

 2    probability theory and theoretical statistics.  By the time 

 3    1967 rolled around, I was just pretty fed up with doing 

 4    theoretical work and I wanted to see what the real world was 

 5    like.

 6         Q.    Can you describe to the court some of the 

 7    consulting activities that you undertook after leaving UCLA?

 8         A.    Yes.

 9               Over the 13 year period, I worked, for example, 

10    on many large air pollution projects for the EPA.  A good 

11    deal of my work was done for the EPA through private 

12    contractors or directly through EPA.

13               For instance, I worked on some large projects on 

14    trying to predict ozone in the Los Angeles basin, national 

15    sulfide oxide trends, formation of carbon monoxide on 

16    freeways, exposure of drivers on surface streets to carbon 

17    monoxide.

18               I reviewed air criterion documents for the EPA 

19    pursuant to revision of the Clean Air Act.  I reviewed and 

20    looked over data concerning monitoring for toxic waste 

21    sites.

22               I reviewed and studied many large computer models 

23    for predicting air pollution.

24               I managed and designed and coordinated two 

25    surveys for EPA on fuel switching.
                                                              2041

 1               Besides EPA, I did work for both the United 

 2    States Department of Transportation and the California 

 3    Transportation Department.  This involved studying freeway 

 4    characteristics, how to improve signalization on surface 

 5    streets, on-ramp control methods for freeways.

 6               There are probably numbers of other things that I 

 7    can't recall offhand.

 8               I did work, for instance, in the legal system.  I 

 9    designed and largely managed and analyzed a survey of the 

10    characteristics of impaneled jurors for the Los Angeles 

11    superior court.

12               I participated, again, in a large scale study of 

13    court recordkeeping by state court systems.  I was an active 

14    participant of a study in the sources of delay in criminal 

15    court proceedings in the Colorado state court system.

16               For The department of Energy, I studied a 

17    large-scale computerized model for predicting energy supply 

18    and demand.  

19               Made extensive studies without much avail of the 

20    stock market.  

21               THE COURT:  You are still poor? 

22               THE WITNESS:  Unfortunately.

23         A.    I'm sure there are many other --  did chemical 

24    work, for instance, trying to recognize toxic substances by 

25    their mass spectra.  
                                                              2042

 1               Did some defense work, for instance, computerized 

 2    recognition of Morse code, trying to recognize ship types by 

 3    remote radar return and so on and so forth.

 4         Q.    And I take it there came a time when you decided 

 5    to go back to academic life at Berkeley?

 6         A.    That's right.

 7         Q.    Why did you decide to join the faculty at 

 8    Berkeley?

 9         A.    In a way that was a hard decision.  I received --  

10    I must say I was really enjoying being a consultant.  I was 

11    working on interesting problems and I liked it.

12               I received a number of offers to go back into 

13    university life, all of which I turned down.

14               Berkeley was different.

15               First of all, there was sentimental reasons.  I 

16    got my Ph.D. there and I maintained connections with a 

17    number of old friends I had there.

18               The second reason is that I would be going back 

19    as an applied person and not a theoretician.

20               And the third reason was it was a great honor.  

21    Berkeley then and now is considered probably the best 

22    statistics department in the country, or at least one of the 

23    two best, and I thought it was a great honor to be invited 

24    to join the faculty, especially after 13 years of almost no 

25    academic publications.
                                                              2043

 1         Q.    Speaking of publications, Professor Breiman, have 

 2    you published any books in the field of statistics?

 3         A.    Yes, I have.

 4         Q.    Could you tell the court about them?

 5         A.    Yes.

 6               While in my theoretical days I wrote a book on 

 7    advanced probability, which is currently being republished 

 8    in a series called classics of mathematics by the Society 

 9    for Industrial and Applied Mathematics.

10               I wrote an undergraduate text on probability 

11    theory which has since been republished, and an 

12    undergraduate text on statistics.

13               Since rejoining the University of California at 

14    Berkeley in 1985, I wrote with some co-authors a book on 

15    applied methodology in statistics.

16         Q.    Professor, have you received any honors in your 

17    profession?

18         A.    Yes, I have.

19         Q.    Could you explain some of them to the court?

20         A.    I'm a Fellow of the Institute of Mathematical 

21    Statistics, which recognizes theoretical work.

22               I am a fellow of the American Statistical 

23    Association in recognition of applied work.

24               A paper I wrote with a co-author in 1985 was 

25    declared the Theory and Methods Paper of the Year by the 
                                                              2044

 1    American Statistical Association.

 2         Q.    When did you become a Fellow in the American 

 3    Statistical Association?

 4         A.    To the best of my recollection, I think it was 

 5    1985.

 6         Q.    Have you also served on any committees of the 

 7    National Academy of Science?

 8         A.    Yes, the Committee on National Statistics.

 9         Q.    Approximately when did you serve on the Committee 

10    on National Statistics?

11         A.    It was in the early eighties.

12         Q.    Professor, if you could take a look at the first 

13    binder in front of you at tab 35, please.

14         A.    Of which exhibit?

15         Q.    Exhibit 35.

16               (Pause)

17               Have you got that?

18         A.    Yes.

19         Q.    Is that a true and correct copy of your 

20    curriculum vitae?

21         A.    Yes, it is.  

22               MR. MILLET:  Your Honor, we offer Defendant's 

23    Exhibit 35.  

24               MR. ZIMROTH:  No objection.  

25               THE COURT:  35 is received.
                                                              2045

 1               (Defendant's Exhibit 35 marked for identification 

 2    was received in evidence.)

 3               MR. MILLET:  Your Honor, at this time we would 

 4    tender Professor Breiman also in the field of statistics and 

 5    ask that he be permitted to express his opinion on matters 

 6    relating to that field.  

 7               MR. ZIMROTH:  Your Honor, I have no doubt that 

 8    Professor Breiman is an expert in some areas of statistics, 

 9    but I haven't heard --  I've heard almost nothing concerning 

10    his expertise in the areas that he is going to testify about 

11    today, and if your Honor please, I would like to ask a few 

12    questions on voir dire.  

13               THE COURT:  Perhaps Mr. Millet should first try 

14    to qualify him in whatever fields you are troubled about.  

15               MR. MILLET:  All right.  

16               THE COURT:  Then you can voir dire.  

17               MR. ZIMROTH:  If I might, according to the letter 

18    and according to what I read in Professor Breiman's report, 

19    he is going to testify about nonsampling error in the PES, 

20    and tis effects on adjustment.

21               May I?  

22               THE COURT:  Go ahead.  I am just nodding that I 

23    understand you.

24    VOIR DIRE EXAMINATION 

25    BY MR. ZIMROTH:
                                                              2046

 1         Q.    Professor Breiman, have you ever written any 

 2    articles on the subject of nonsampling error in the PES?

 3         A.    No.

 4         Q.    Have you ever written any articles on the subject 

 5    of nonsampling error in the PEP?

 6         A.    No.

 7         Q.    You know what the PEP was?

 8         A.    Yes.

 9         Q.    Okay.

10               Have you ever written any articles on the 

11    measurement of nonsampling error in any sample survey?

12         A.    No.

13         Q.    Prior to your retention by the government in this 

14    case, have you ever conducted any research on the topic of 

15    nonsampling error in sample surveys?

16         A.    Other than my concern about nonsampling errors in 

17    the surveys that I ran, no.

18         Q.    Do you recall being asked that same question at 

19    your deposition?

20         A.    No.

21         Q.    Let me see if this refreshes your recollection, 

22    it's on page 8.  

23               MR. MILLET:  Could the witness be shown a copy?  

24               MR. ZIMROTH:  This is a very simple question and 

25    answer, your Honor.  
                                                              2047

 1               THE COURT:  If it's short, go ahead.

 2         Q.    "Q.   Have you written any articles about the 

 3    measurement of nonsampling error in sample surveys?

 4               "A.    No.

 5               "Q.    Have you conducted any research on that 

 6    topic?

 7               "A.    No." 

 8               Do you recall being asked those questions and 

 9    giving those answers?

10         A.    You know, I was asked a lot of questions on 

11    deposition.  If that's there, I'm sure that's what I said.

12         Q.    Okay.

13               Have you ever written any articles on any aspect 

14    of the decennial census?

15         A.    No.

16         Q.    Your resume lists a series of invited talks.

17         A.    Correct.

18         Q.    Have any of those been on the subject of 

19    nonsampling error in the PES?

20         A.    No.

21         Q.    Or on any aspect of the decennial census?

22         A.    No.

23         Q.    You mentioned that you were a member of the 

24    Committee on National Statistics, correct?

25         A.    That's right.
                                                              2048

 1         Q.    You are aware, aren't you, that the Committee on 

 2    National Statistics does work through panels; is that 

 3    correct?

 4         A.    That's right.

 5         Q.    And there was a panel that studied the PES and 

 6    the PEP, is that correct?  What was the name of the panel? 

 7               The panel on decennial census methodology.

 8         A.    Yes, there was such a panel.

 9         Q.    And you were not a member of that panel, were 

10    you?

11         A.    That's right.

12         Q.    Okay.

13               Have you ever been qualified to testify as an 

14    expert in a judicial proceeding on the subject of 

15    nonsampling error in sample surveys?

16         A.    No.

17         Q.    Or on the PES or the PEP?

18         A.    That's correct.

19         Q.    Or on any aspect of the decennial census?

20         A.    That's correct.

21         Q.    Now, in your entire career, you had experience in 

22    the design and implimentation of three sample surveys, isn't 

23    that right?

24         A.    That is correct, except that I also served as a 

25    reviewer for the EPA for numerous proposed sample surveys.
                                                              2049

 1         Q.    And on the three that you actually had experience 

 2    in designing and implementing, those were during the 1970s, 

 3    isn't that right?

 4         A.    Yes.

 5         Q.    And was any of them a national survey?

 6         A.    No.  The largest one was fuel switching, which 

 7    encompassed five states in the northwest.

 8         Q.    And that had to do with catalytic converters, 

 9    right?

10         A.    The use of gasoline.

11         Q.    The use of gasoline in automobiles?

12         A.    Right.

13         Q.    And one had to do with juror characteristics?

14         A.    I didn't hear you.

15         Q.    The other had to do with juror characteristics?

16         A.    Impaneled juror characteristics.

17         Q.    And that was done approximately 20 years ago?

18         A.    I think that's about right.  

19               MR. ZIMROTH:  Your Honor, I have two quite 

20    fundamental objections to Professor Breiman's testimony here 

21    today.

22               One is, as I understand from his report, he is 

23    going to be testifying in two respects:

24               One is that he is going to be explicating what 

25    the Census Bureau did prior to July 15 on the question of 
                                                              2050

 1    nonsampling error, and with respect to that question, it is 

 2    perfectly clear that Professor Breiman is not an expert, and 

 3    I find it, frankly, bizarre that the defendants on this 

 4    subject, when they have available to them the world's 

 5    premier experts in the Census Bureau, come in with Professor 

 6    Breiman.

 7               That is the first point.

 8               The second point I think is equally important, 

 9    because in addition to simply explicating what the Census 

10    Bureau did prior to July 15, Professor Breiman, according to 

11    the report that we have been given, is going to be 

12    testifying about work that he did and findings that he made 

13    after July 15, 1991.

14               Now, when this issue was raised about work that 

15    was done post-July 15, 1991, your Honor overruled our 

16    objection and ruled that the defendants could put in 

17    evidence with regard to the analysis of census data 

18    conducted by the Census Bureau staff since July 15, 1991 

19    primarily through the testimony of Peter Bounpane and Robert 

20    Fay, whom you have already heard last week.

21               Professor Breiman's testimony is of an entirely 

22    different character.  This is not work that has been done by 

23    the Census Bureau, it was not before the Secretary on July 

24    15, it is not --  to the extent that the Census Bureau has 

25    accepted that work, you have already heard the testimony of 
                                                              2051

 1    Dr. Fay and Mr. Bounpane, and so I think that this testimony 

 2    is well beyond the scope of your Honor's ruling. 

 3               THE COURT:  Finito?  

 4               MR. ZIMROTH:  Finito.  

 5               MR. MILLET:  My turn?  

 6               THE COURT:  Mr. Millet.  

 7               MR. MILLET:  Your Honor, quite frankly, we 

 8    thought it might be somewhat refreshing for the court to 

 9    hear from a witnesses whose professional career is not tied 

10    to the outcome of this case, as have all the plaintiffs' 

11    witnesses as to Professor Breiman's prior expense in census 

12    matters, that really has nothing to do with his ability to 

13    analyze data in this case and present a professional opinion 

14    based upon what I think is quite an impressive set of 

15    credentials and experience.

16               Mr. Zimroth's objection, if anything, goes to 

17    whatever weight the court may decide to place on Professor 

18    Breiman's analysis and not to his qualifications or the 

19    admissibility of his opinions.

20               The post-July 15 issue has already been resolved, 

21    it was resolved in defendants' favor and the court indicated 

22    that you were going to apply a heightened scrutiny to 

23    post-July 15 analysis.  Your Honor, we welcome that.

24               If you would like, I will go on and ask Professor 

25    Breiman further questions to lay a foundation for the work 
                                                              2052

 1    that he has done in preparing his analysis in this case, but 

 2    certainly his qualifications as an expert cannot be 

 3    questioned.  

 4               THE COURT:  Well, he has admitted that he has 

 5    does precious little work in nonsampling error.  I'm a 

 6    little troubled by that.

 7               If that is the core of his testimony, I'm deeply 

 8    troubled.  

 9               MR. MILLET:  Then if the court will permit me, I 

10    will ask some further questions.  

11               THE COURT:  That's what I suggested 20 minutes 

12    ago.  

13               MR. ZIMROTH:  Your Honor, may I just interrupt 

14    for one second?  

15               THE COURT:  Yes.  

16               MR. ZIMROTH:  Before Mr. Millet continues, I 

17    would like to hand up to the court a copy of your Honor's 

18    decision on this post-July 15 issue, because I think Mr. 

19    Millet's characterization is not accurate.  I think it is 

20    clear that you are talking there about work that is done by 

21    the Bureau.  

22               MR. MILLET:  And I would remind Mr. Zimroth that 

23    that was based on a motion directed to post-July 15 work by 

24    the Census Bureau in which they filed which they sought to 

25    exclude at the time they were well aware that Professor 
                                                              2053

 1    Breiman was prepared to testify as well as our own experts 

 2    were prepared to testify to other matters.  

 3               MR. ZIMROTH:  You know, your Honor, there is a 

 4    fundamental legal issue here about this man's testimony 

 5    beyond his expertise.

 6               What you are going to hear, if you allow this 

 7    testimony, is purely post-hoc rationalization.  Nothing that 

 8    he says about his work since July 15 was before either the 

 9    Secretary or the Census Bureau at that time, so it's --  

10               THE COURT:  But if we are trying to determine 

11    whether the Secretary was arbitrary and capricious, would it 

12    not be helpful to get the opinion of an expert, assuming we 

13    can qualify him as such, that he looked over the whole 

14    situation even after the fact and these are what he found 

15    and these were his conclusions?  

16               MR. ZIMROTH:  Well, I think, your Honor, the law 

17    is pretty clear on this subject that in judging the 

18    arbitrariness and capriciousness of the Secretary's 

19    decision, what has to be looked at is what was before the 

20    Secretary and that post-hoc rationalizations simply are not 

21    relevant.

22               THE COURT:  I'm not sure we are saying anything 

23    different.

24               I am saying that you are absolutely right, you 

25    have to base this on what the Secretary knew and made a 
                                                              2054

 1    judgment on, but what he knew as a matter of expert opinion, 

 2    and that's where I think he may be helpful.  

 3               MR. ZIMROTH:  Well, I don't think that what he 

 4    knew could possibly be anything that Professor Breiman did 

 5    after July 15, 1991.  I mean, that's an impossibility.  

 6               THE COURT:  He is like a detective who went in 

 7    after the homicide.  He is now telling me what he found.  

 8               MR. ZIMROTH:  There is one other --  well, let's 

 9    see if Mr. Millet can establish the expertise, because I 

10    think that is a fundamental point, but I would like to 

11    continue this if I might after Mr. Millet.  

12               THE COURT:  All right.  

13               MR. MILLET:  If I can just say one more thing in 

14    response before I continued questioning, I would point your 

15    Honor's attention to the Sixth Circuit decision, Young 

16    versus Klutznick, another of the 1980 adjustment cases where 

17    the Sixth Circuit took judicial notice for the first time on 

18    appeal of new analyses that were developed after the 

19    adjustment decision.  

20               MR. ZIMROTH:  And in that matter, I would like to 

21    refer the court to a recent case of the Second Circuit 

22    called Immigration and Naturalization Service, Vargas 

23    against the Immigration and Naturalization Service, which is 

24    cited at 938, Fed. 2d 358 at page 363, which says, "An 

25    administrative agency --  " a quote from another decision, 
                                                              2055

 1    "An administrative agency's decision, unlike those of a 

 2    district court, cannot be sustained on the ground appearing 

 3    in the record to which the agency made no reference.  To the 

 4    contrary, the board's decision stands or falls on its 

 5    express findings and reasoning."  

 6               MR. MILLET:  Your Honor, Mr. Zimroth knows very 

 7    well that the Secretary referred to issues and bias and 

 8    nonsampling error in his decision.  

 9               THE COURT:  Let me hear the qualification.  I am 

10    troubled more by that.  

11               MR. MILLET:  Certainly.  

12    BY MR. MILLET:

13         Q.    Professor Breiman, let's go back and talk some 

14    more about your work as a consultant before you went back to 

15    academic life in Berkeley.

16               In your work as a consultant, did you have 

17    occasion to review the results of sample surveys for data 

18    quality?

19         A.    Yes, I did.

20         Q.    Can you explain some of those to the court, 

21    please?

22         A.    Well, for instance, I was used by the EPA to 

23    review sample surveys both for the design and in particular 

24    the possibilities of nonsampling error coming up in those 

25    designs.
                                                              2056

 1               I think as I recall I reviewed the results of 

 2    surveys that were carried out by the EPA for various reasons 

 3    and gave EPA opinions on whether the results were sound or 

 4    not.

 5               I certainly, as I say, was very cognizant and 

 6    conscious of the possibilities of nonsampling error in each 

 7    of the surveys that I designed and worked very hard to think 

 8    about all possible sources and keep them at a minimum.

 9         Q.    Did you have any occasion to review in particular 

10    the results of statistical analysis as they related to 

11    statistical models concerning auto emissions?

12         A.    Oh, yes.  Oh, yes.

13         Q.    Could you explain that to the court, please?

14         A.    Well, I reviewed many, many large-scale 

15    statistical models.

16               If you want a particular example or examples, 

17    I'll be glad to give you one.

18         Q.    Could you give one for the court just to 

19    illustrate the point?

20         A.    Well, this particular example I found somewhat 

21    amusing because of my attitude toward very large statistical 

22    models and complicated statistical models.

23               This was --  

24               THE COURT:  Nice to know there is a little humor 

25    in statistics.  I have seen precious little of it.
                                                              2057

 1         A.    Well, I found this funny.

 2               Ethyl Corporation which manufactures and an 

 3    additive that gets put into gasoline, at one point they 

 4    couldn't put lead in any more, so they tried to figure out 

 5    how to make a different additive, and in order to get 

 6    permission from the EPA you had to do a test, so they took 

 7    30 or 40 cars and ran them around a track for 50,000 miles 

 8    and kept track of how much emissions were coming out of the 

 9    cars.

10               And then they submitted to EPA what I call a 

11    marvelous example of edifice buildings.  It was about 150 

12    pages of statistical works, I computed no less than 126 

13    regression equations.  They were talking about outliers, 

14    they were talking about robustness, everything.  It was a 

15    marvelous piece of work.

16               And then I asked EPA to supply me with the raw 

17    data, and I found that what these people had managed to do 

18    with their edifice building was to completely disappear 

19    about one-third to one-half of the actual emissions.

20         Q.    Did your experience as a consultant lead you to 

21    adopt any particular approaches to statistical analysis?

22         A.    Yes, it did.

23         Q.    Could you explain that to the court, please?

24         A.    This is summarized in a 1984 paper.

25               I was invited to submit a paper on statistics and 
                                                              2058

 1    public policy because of the sort of varied experience I 

 2    had, and the title of the paper is Nail Finders, Edifices 

 3    and Oz.  

 4               THE COURT:  Is it listed here in your CV? 

 5               THE WITNESS:  Yes, it, your Honor.  

 6               THE COURT:  Do you happen to know where?  

 7               MR. MILLET:  I believe you will find it under 

 8    number 10 under recent publications, Judge, the fifth page.

 9         A.    And what I say in that paper is a statistician 

10    has two important jobs:

11               The first one is to figure out what the real 

12    problem is, and the second one is to figure out if the data 

13    can provide an answer to that problem.

14               The title of the paper, Nail Finders, Edifices 

15    and Oz, refers to the following fact, that Nail Finders 

16    comes out of an old saying give a man a hammer and every 

17    problem looks like a nail, and what I was referring to there 

18    is that we teach our students in statistics a certain body 

19    of knowledge and, by God, they go out in the world and no 

20    matter what problem they look at, they are going to make it, 

21    say, a test of the hypothesis.

22               Edifices refers to an unfortunate tendency I 

23    found in many of my colleagues to create just these large 

24    superstructures of statistical techniques based on 

25    unwarranted assumptions and piling equations on equations 
                                                              2059

 1    with very little worry about whether the assumptions are 

 2    true or whether the data underneath the assumptions is of 

 3    high quality or not.

 4               And Oz refers to the tendency, again, in 

 5    statistics to try to dazzle and bewitch the nonstatisticians 

 6    by the use of many statistical technical terms.

 7         Q.    You used the term data quality.  In your mind, is 

 8    that a similar term to nonsampling error?

 9         A.    Yes, exactly.

10               One thing I've learned through all my consulting 

11    is this, and this is what I tell my students:

12               The first thing you got to worry about is the 

13    quality of the data.  If the data isn't good, you're not 

14    going anywhere.

15         Q.    Professor Breiman, did you undertake any review 

16    of the post-enumeration survey in this case?

17         A.    I certainly did.

18         Q.    Could you please describe that to the court?

19         A.    Yes.  I undertook my review.  The basic 

20    information I used were the P studied issued by the census.  

21               MR. ZIMROTH:  Excuse me, your Honor.  I thought 

22    we were trying to establish Professor Breiman's --  

23               MR. MILLET:  I am attempting to lay the 

24    foundation.  

25               THE COURT:  I think we are getting too deep in 
                                                              2060

 1    the merits.  

 2               MR. MILLET:  I was going to stop simply after 

 3    describing what work he did to lay a foundation.  

 4               THE COURT:  That is an objection I will sustain.  

 5               MR. ZIMROTH:  Thank you, your Honor.  

 6               MR. MILLET:  Your Honor, I will again tender 

 7    Professor Breiman as an expert in the field of statistics.  

 8               THE COURT:  In your teaching on statistics at 

 9    whatever level, graduate, undergraduate, do you address this 

10    subject of nonsampling error? 

11               THE WITNESS:  Not specifically, your Honor.  

12               THE COURT:  Do you have any further questions, 

13    Mr. Zimroth?  

14               MR. ZIMROTH:  Just --  not questions, but to 

15    point out that whatever experience Dr. Breiman has testified 

16    in his consulting has to do with EPA auto catalytic 

17    converters, traffic flow and the like and not with human 

18    populations, not with the census, not with the PES, not with 

19    the PEP, and all the issues that are covered in Professor 

20    Breiman's report has no experience with any of that.  

21               MR. MILLET:  Your Honor, if I may --  

22               THE COURT:  Do you have a copy of that report?  

23               MR. MILLET:  Yes, your Honor.  It is under 

24    tab 36.

25               Your Honor, if I may, I seem to recall that one 
                                                              2061

 1    of the plaintiffs' main witnesses testified at great length 

 2    about his extensive experience with fire alarms and false 

 3    alarms.

 4               Your Honor, Professor Breiman has reviewed the 

 5    PES for data quality, he has testified that he spent a great 

 6    deal of his career in the real world dealing with surveys 

 7    and data quality.  That's the examination he has done.

 8               As I said, we thought that the court might like 

 9    to hear from someone whose career is not tied to the outcome 

10    of this case.  

11               MR. ZIMROTH:  That is outrageous that you would 

12    keep saying that, Mr. Millet.  

13               THE COURT:  I think I have heard enough to make a 

14    ruling.

15               Let me take a short break and, curiously enough, 

16    thing.

17               (Recess) 

18               THE COURT:  Under Rule 702, expert testimony is 

19    admissible when two conditions are met:

20               First, it must deal with scientific or other 

21    specialized knowledge which will assist the trier of the 

22    fact, me, to understand the evidence, and secondly, it must 

23    be presented by a witness qualified as an expert by 

24    knowledge, skill, experience and the like.

25               Without meaning any disrespect at all to Dr. 
                                                              2062

 1    Breiman, the thrust of his testimony as outlined in 

 2    Defendant's Exhibit 36 is to point out nonsampling type 

 3    errors in both the PES and the PEP.  

 4               While Dr. Breiman is qualified, indeed, highly 

 5    qualified to explain the theoretical construct that 

 6    underlies the statistics here, he could explain the fish in 

 7    the pond problem again, the fact remains he has never tagged 

 8    any fish and, therefore, I don't believe that he is 

 9    qualified as an expert and I would not find the testimony 

10    particularly helpful where he attempts to testify to faults 

11    that he finds in the tagging of the fish process which 

12    conceptually is what this is all about.

13               Accordingly, I will not permit any testimony to 

14    be elicited from Dr. Breiman about nonsampling error in 

15    either the PES or the PEP.

16               If you have any other type of expert testimony 

17    that you think might be helpful, I will be glad to hear it.

18               (Continued on the next page) 

19    

20    

21    

22    

23    

24    

25    
                                                              2063

 1               MR. MILLET:  First of all, your Honor, Professor 

 2    Breiman was not going to testify to anything having to do 

 3    with PEP.  I was attempting, before we finished our 

 4    questioning, to lay a foundation for Professor Breiman's 

 5    review of the data, and particularly the P studies, and for 

 6    an evaluation and explanation of that data.  If the Court 

 7    would find that helpful in considering, again, Professor 

 8    Breiman's qualifications, I would be happy to resume that.  

 9    If the Court finds that that would not be helpful, then I 

10    would ask to make a proffer.  

11               THE COURT:  Make your proffer.  
                                                 

12               MR. MILLET:  Your Honor, our proffer is that if 

13    Professor Breiman were permitted to testify, he would 

14    testify to contents of his report marked for identification 

15    purposes as Defendants' Exhibit 36.  Professor Breiman would 

16    testify to his review and analysis of the sources of 

17    nonsampling error in the PES, particularly P studies.  

18               Professor Breiman would testify to his views on 

19    the level of nonsampling error and data quality in the PES 

20    based upon his experience as a statistician and his 

21    philosophy of statistical analysis, which he did testify to, 

22    resulting from his long experience as a consultant in the 

23    statistical field.  

24               Professor Breiman would testify to numerous areas 

25    of concern regarding data quality in the post-enumeration 
                                                              2064

 1    survey.  In particular, Professor Breiman would testify to 

 2    substantial problems revealed by the the data concerning 

 3    matching errors.  

 4               Professor Breiman would testify that in a 

 5    rematching study conducted by the Census Bureau, substantial 

 6    disagreement rates were revealed between two different teams 

 7    of matchers, indicating that matching, when the had you map 

 8    element was involved, involves a great deal of judgment and 

 9    is not necessarily scientific.  That would be true on both 

10    the E sample and P sample side.  

11               Professor Breiman would also testify to his 

12    review of fabrications in the PES, in particular to the 

13    three conflicting studies done by the Census Bureau on the 

14    fabrication level in the post-enumeration survey.  

15               Professor Breiman would also testify to the 

16    presence of census day address errors in the post- 

17    enumeration survey, essentially persons who moved into the P 

18    sample blocks after census day but before the post- 

19    enumeration survey was conducted.  He would testify to how 

20    that introduces a measure of bias in the post-enumeration 

21    survey results.  

22               Professor Breiman would also testify to the 

23    problem of geocoding errors, which is a term used by the 

24    Census Bureau to apply to the efforts to match P sample 

25    respondents to census results outside the PES sample block.  
                                                              2065

 1               Professor Breiman would also testify to data 

 2    quality in the evaluation follow-up, where there was a 

 3    substantial amount, in his view, of rejected interviews in 

 4    the evaluation follow-up when the evaluation follow-up was 

 5    to be conducted by the Bureau's most experienced personnel.  

 6               Dr. Breiman would also testify to missing data 

 7    rates in the post-enumeration survey and to his analysis of 

 8    the missing data rates, also to the fact that missing data 

 9    rates, after his study, are shown to be highly correlated to 

10    the undercount estimates in the post-enumeration survey.  

11               Dr. Breiman would also testify as to his opinion 

12    as to quality of the imputation model used in the post- 

13    enumeration survey.  In particular, he would testify to his 

14    study of a comparison of the evaluation follow-up rematch 

15    results with the matched probabilities generated by the 

16    imputation model, and would testify in his opinion that 

17    there is no correlation between the matched probabilities 

18    determined by the imputation model and the actual rematching 

19    done in the evaluation follow-up, and as a result of that he 

20    would testify that the imputation model is seriously flawed.  

21               Professor Breiman would also testify that in the 

22    data quality indicators which he examined, indications of 

23    bad data are highly correlated with undercount, indicating 

24    that the data concerning undercount may not be reliable.  

25               Professor Breiman would also add up following his 
                                                              2066

 1    analysis all the areas of nonsampling error, including a 

 2    substantial number of nonsampling error that were not 

 3    included in the Bureau's P-16 study referred to often in 

 4    this case so far as the total error model.  He would testify 

 5    that there were substantial amounts of nonsampling error 

 6    that were not included in the total error model upon which 

 7    plaintiffs' witnesses have relied in this case 

 8    substantially.  

 9               He would testify that if all the areas of 

10    nonsampling error, both those in P-16 and the additional 

11    ones that were omitted from P-16, were taken together, that 

12    would indicate a reduction in the net national undercount by 

13    over four million people.  He would testify that in his 

14    opinion that is, of course, not an indication of what the 

15    net national undercount is but is a very strong piece of 

16    evidence indicating the unreliability of the post- 

17    enumeration survey as a tool to measure undercount in this 

18    case.  

19               Dr. Breiman would also testify that taking all 

20    the areas of nonsampling error which he has reviewed, 

21    including those not included in the Bureau's total error 

22    model, there would be a decrease in the undercount in 

23    minority evaluation poststrata; in other words, your Honor, 

24    that the nonsampling error has a greater effect in 

25    overstating the differential undercount by minority groups.  
                                                              2067

 1               He would testify as to his expectations of other 

 2    areas of nonsampling error that remain to be studied by the 

 3    Bureau.  He would also testify that there is no reliable 

 4    measure in this case for correlation bias and therefore it 

 5    is not an appropriate consideration.  

 6               THE COURT:  I wish to compliment counsel.  That 

 7    is the finest offer of proof I have ever heard, certainly 

 8    the most meticulous one.  But I have heard nothing to change 

 9    my mind.  I think you have made your record.  

10               Is there anything else that you want to call Dr. 

11    Breiman for?  

12               MR. MILLET:  No, your Honor.  At this point I 

13    would propose we adjourn for a few minutes.  

14               THE COURT:  How long would you like?  

15               MR. MILLET:  How about 20 minutes?  

16               THE COURT:  Fine, 20 minutes.  

17               (Recess)

18               THE COURT:  Mr. Baron, you may call your next 

19    witness. 

20               MR. BARON:  I call Kenneth Wachter.  

21    KENNETH WACHTER, 
                       

22         called as a witness by the defendants, having

23         been duly sworn, testified as follows: 

24               THE COURT:  State your full name and spell your 

25    last name for the record.  
                                                              2068

 1               THE WITNESS:  Kenneth Wilcox Wachter, W A C H T E R.  

 2    DIRECT EXAMINATION 

 3    BY MR. BARON:

 4         Q.    Good morning, Professor Wachter.  

 5               MR. BARON:  Your Honor, do you have a copy of 

 6    defendants' exhibit book where Defendants' Exhibit 38 is, 

 7    Professor Wachter's CV?  

 8               THE COURT:  I believe I have it right in front of 

 9    me here.  I do.

10         Q.    Professor Wachter, where are you currently 

11    employed?

12         A.    I am a professor of demography and statistics at 

13    University of California, Berkeley.

14         Q.    How long have you been at Berkeley?

15         A.    I have been at Berkeley since 1977.

16         Q.    What courses in statistics have you taught at 

17    Berkeley?

18         A.    I have taught both the theoretical and the 

19    applied multivariate statistics courses at the graduate 

20    level, and I have taught stochastic processes at the 

21    graduate level, applied stochastic processes.

22         Q.    What courses in demography have you taught at 

23    Berkeley?

24         A.    I teach most of the demographic methods courses.  

25    The Berkeley demography department is something of a center 
                                                              2069

 1    for demographic methods.

 2         Q.    Do you teach at the graduate level in that 

 3    department?

 4         A.    I teach both at the undergraduate and graduate 

 5    levels.

 6         Q.    Could you please suprise for the court your 

 7    educational background?

 8         A.    I have a BA from Harvard, an MA in applied 

 9    mathematics, a specialty degree, from Oxford, and my Ph.D is 

10    in statistics from Cambridge.  I worked under Professor 

11    David Kendall and John Kingman, now Sir John Kingman, at 

12    Cambridge.

13         Q.    Are you a fellow of the American Statistical 

14    Association?

15         A.    Yes.

16         Q.    Are you an editor of a professional journal?

17         A.    Yes.  I am one of the editors of Mathematical 

18    Population Studies. 

19         Q.    Could you highlight for the Court some of the 

20    committees on which you have served in a professional 

21    capacity?

22         A.    From 1985 to 1991 I was a member of the Committee 

23    on National Statistics of the National Research Council.  

24    This is the parent committee for the Decennial census Panel 

25    that we have heard a good deal about in the court.  
                                                              2070

 1               I also served on the Panel on Immigration 

 2    Statistics and on the Special Working Group for the Survey 

 3    of Income and Program Participation for the Committee on 

 4    National Statistics. 

 5               Before that I served for, I guess, four years on 

 6    the National Science Foundation Panel on Measurement Methods 

 7    and Data Improvement.  This is the part of the National 

 8    Science Foundation that funds applied statistics and, in 

 9    particular, survey research.  So although I do not 

10    specialize in survey research, I was responsible for giving 

11    advice on funding decisions for a large part of the survey 

12    research that was undertaken under NSF support in the 1980s, 

13    and I have a fairly broad, general familiarity with survey 

14    work from that experience.

15         Q.    You also served on the special advisory panel to 

16    the Secretary of Commerce on the 1990 Decennial census?

17         A.    That's true.

18         Q.    Have you published any books in the areas of 

19    demography or statistics?

20         A.    Yes.  I have published two books and a couple of 

21    edited volumes.  These books are mainly in statistics rather 

22    than in demography.

23         Q.    Was one book concerning height?

24         A.    Yes.  With some coauthors, I just published a 

25    statistical study of data on heights in Britain over two 
                                                              2071

 1    centuries.  

 2               THE COURT:  That is pretty tall.  

 3               THE WITNESS:  It is.

 4         Q.    Would you summarize for the Court what areas of 

 5    interest you have published in apart from these two books?

 6         A.    I would say my main areas of specialization are 

 7    in statistical computer experiments, what is called 

 8    simulation experiments.  I developed over close to 20 years 

 9    now a demographic statistical computer simulation program 

10    called SocSim, which has been applied to many problems in 

11    demography.  

12               I also have a specialization in the statistical 

13    modeling of demographic heterogeneity from area to area.  

14    Beyond that I work in demographic mathematical demography, 

15    feedback models.  I work in statistical data analysis, 

16    particularly historical data analysis.  

17               More recently I have worked in statistical 

18    astronomy, studying data from the remote cosmos.  And over 

19    the last five years or so I have studied questions of census 

20    adjustment and census accuracy

21         Q.    Have you received any awards in your professional 

22    career to date?

23         A.    I guess the most important award is the Mendel 

24    Schepps award, which is one of the two highest honors of the 

25    Population Association of America.  This award is given for 
                                                              2072

 1    contributions to mathematical demography and demographic 

 2    methodology.  I received it four years ago.

 3         Q.    Did you testify an as expert witness in the 

 4    defendants in the Cuomo v. Baldridge litigation involving 

 5    the 1980 census?

 6         A.    I did.

 7         Q.    When did you begin your service with the Special 

 8    Advisory Panel on the 1990 Decennial census?

 9         A.    Effectively in October 1989. 

10         Q.    Could you give the Court the nature of your 

11    involvement while serving on the Special Advisory Panel?

12         A.    Serving on the Special Advisory Panel was a 

13    considerable task.  It involved meeting and advising the 

14    Department of Commerce on the original guidelines, making 

15    very detailed suggestions for revisions.  It then involved 

16    keeping abreast of Census Bureau plans, plans for the post- 

17    enumeration survey and for the evaluations, giving advice 

18    where called for.   

19               I also accompanied a census enumerator in 

20    Oakland, and I accompanied a post-enumeration survey follow- 

21    up interviewer, and I went twice to the San Diego processing 

22    office to observe different stages of matching.  

23               THE COURT:  How long did you accompany the 

24    enumerator?  

25               THE WITNESS:  For one day.  
                                                              2073

 1               THE COURT:  And the post?  

 2               THE WITNESS:  The same.

 3         Q.    Did you also appear at one or more Congressional 

 4    hearings on census adjustment as a Special Advisory Panel 

 5    member?

 6         A.    I submitted written testimony to one hearing of 

 7    the House Subcommittee on census and Population, and I 

 8    appeared in person at three or four more hearings, one of 

 9    which was joint with the Senate Committee on Governmental 

10    Regulation.

11         Q.    Can you give the Court an estimate of the time 

12    you spent working on Special Advisory Panel related issues?

13         A.    Well, averaging over the whole period, it 

14    probably, for the most part, averaged one to two days a week 

15    up to about the beginning of April of 1991, when the data 

16    from the PES began to arrive in usable form.  From April 

17    until the 17th of June 1991, when I mailed my report to the 

18    Secretary, basically I taught my courses, I saw my doctoral 

19    students, and then I worked night and day on Census Bureau 

20    data.  

21               MR. BARON:  At this time I move the admission of 

22    Defendants' Exhibit 38, which constitutes Professor 

23    Wachter's curriculum vitae.  

24               THE COURT:  Hearing no objection, 38 is admitted. 

25               (Defendants' Exhibit 38 for identification was 
                                                              2074

 1    received in evidence) 

 2               MR. BARON:  At this time I proffer Dr. Wachter as 

 3    an expert on demography and statistics.  

 4               MR. GOLDIN:  Could I have some very brief voir 

 5    dire, your Honor?  

 6               THE COURT:  Yes. 

 7    VOIR DIRE EXAMINATION 

 8    BY MR. GOLDIN:

 9         Q.    Dr. Wachter, you testified a moment ago, as I 

10    understood your testimony, that in connection with your 

11    service on the Committee on National Statistics you gave 

12    advice concerning funding of certain survey research 

13    projects, is that correct?

14         A.    No.  There are two groups.  It is the National 

15    Science Foundation Panel on Measurement Methods and Data 

16    Improvement which reviews proposals to the National Science 

17    Foundation.  That is the context in which I reviewed 

18    proposals over the range of applied statistics, but there is 

19    a special concentration of that panel on survey methods.

20         Q.    It is in connection with that work that you have 

21    given advice on funding of survey research projects?

22         A.    Along with the other members of the panel, that's 

23    right.

24         Q.    That is the principal basis for your having 

25    familiarity with survey research, is that correct?
                                                              2075

 1         A.    Yes.  I have read 200 proposals for projects in 

 2    survey research and engaged in discussions of their faults 

 3    and virtues, and that is my principal source of knowledge of 

 4    the field.

 5         Q.    You have not conducted any research in the field 

 6    of survey research yourself, is that right?

 7         A.    I have conducted long-standing research into 

 8    questions of census adjustment, but I haven't -- independent 

 9    of my interest in the census and the PES and of my 

10    involvement on the Committee on National Statistics Working 

11    Group on the Survey of Income and Program Participation, I 

12    haven't conducted specialized research.  

13               I do teach an undergraduate course which deals 

14    with a number of problems and issues in surveys.  But those 

15    are at an undergraduate level.

16         Q.    You have not yourself conducted any surveys, is 

17    that right?

18         A.    That's right.

19         Q.    In your curriculum vitae Defendants' Exhibit 38, 

20    under research interests, you do not list survey research as 

21    an interest of yours, is that right?

22         A.    That's right, yes.

23         Q.    Do you consider yourself an expert in survey 

24    research?

25         A.    No.  I have a general familiarity, but I'm not a 
                                                              2076

 1    specialist in survey research.  I am not an expert in survey 

 2    research, at least except as it is involved in census 

 3    adjustment.  

 4               MR. GOLDIN:  Your Honor, with the understanding 

 5    that Dr. Wachter is being proffered not as an expert in 

 6    survey research, we have no objection.  

 7               MR. BARON:  Your Honor, I do believe that 

 8    Professor Wachter meets the standard in terms of his general 

 9    involvement in census adjustment issues, and his teaching an 

10    undergraduate source in survey methods, meets the same 

11    standard as an expert in survey research as, for example, 

12    Professor Estrada did as a statistician, given his 

13    background of teaching one basic course in statistics.  

14               So I again proffer Professor Wachter without 

15    exception as an expert in demography and statistics.  

16               MR. GOLDIN:  Your Honor, Dr. Wachter just 

17    testified that survey research was not a particular area of 

18    interest of his.  He disclaimed -- 

19               THE COURT:  He didn't say not an area of 

20    interest.  

21               MR. GOLDIN:  He said that he agreed that it was 

22    not one of the areas of research interest that has been 

23    identified in his curriculum vitae.  He disclaimed expertise 

24    in the area.  It is apparent that he has not himself 

25    conducted surveys.  I don't think that the proffer of Dr. 
                                                              2077

 1    Wachter as an expert, therefore, ought to encompass the area 

 2    of survey research.  

 3               THE COURT:  I am comfortable with accepting Dr. 

 4    Wachter's testimony as an expert in demographics, 

 5    statistics, including survey research.  

 6               MR. BARON:  Thank you, your Honor.  

 7    BY MR. BARON:

 8         Q.    Professor Wachter, are you being compensated for 

 9    your services as an expert in this litigation?

10         A.    Yes.

11         Q.    Do these services include undertaking independent 

12    analysis of the PES and demographic analysis data?

13         A.    Yes.  Extensive analysis and independent 

14    research.

15         Q.    Professor Wachter, do you believe that an 

16    undercount exists?

17         A.    In the 1990 census?

18         Q.    Yes.

19         A.    Yes, I do.

20         Q.    Do you believe that differential undercounts 

21    exist in the 1990 census?

22         A.    Yes, I do.

23         Q.    Do you have any doubts that differential 

24    undercounts are serious?

25         A.    I have no doubts whatever that the differential 
                                                              2078

 1    undercounts in the 1990 census are serious.  What I do have 

 2    doubts about is whether adjusted counts can be shown to 

 3    improve upon unadjusted counts.  That is the issue posed in 

 4    the guidelines, and that issue seems to me very much in 

 5    doubt.

 6         Q.    Did you analyze the relative accuracy of the 

 7    adjusted and unadjusted counts in your recommendation to the 

 8    Secretary?

 9         A.    I did.

10         Q.    Is Defendants' Exhibit 39 the report setting 

11    forth your recommendation to the Secretary on the adjustment 

12    issue?  

13               MR. BARON:  Your Honor, this had has already been 

14    received into evidence as part of Defendants' Exhibit 1.

15         A.    Yes.

16         Q.    Among the topics discussed in your report, do you 

17    discuss the Bureau's findings based on demographic analysis 

18    as such findings relate to the findings from the PES?

19         A.    I do.

20         Q.    Do you also discuss the homogeneity assumption as 

21    embodied within the 1990 PES methodology?

22         A.    I do.

23         Q.    Let's first to the turn to the topic of 

24    demographic analysis.  Dr. Wachter, are you familiar with 

25    the census Bureau's 1990 Program of Demographic Analysis?
                                                              2079

 1         A.    Yes.

 2         Q.    What is demographic analysis?

 3         A.    Demographic analysis is an approach for obtaining 

 4    population estimates independent of the results of a census, 

 5    using administrative data like birth and death certificates.

 6         Q.    Why are the results from demographic analysis 

 7    important?

 8         A.    Demographic analysis provides the main external 

 9    check on the census and on the PES and is, therefore, 

10    potentially one of the main sources for judging the relative 

11    numeric accuracy of adjusted and unadjusted counts at the 

12    national level.

13         Q.    Are there specific limitations to demographic 

14    analysis?

15         A.    The first main limitation is the one I just 

16    mentioned.  Demographic analysis is confined to estimates at 

17    the national level.  There is no information from the 

18    Bureau's demographic analysis about populations of states, 

19    cities, regions, local areas.  It is confined to the 

20    national level.  

21               The other main limitation is that it is confined 

22    to a very stylized distinction between blacks and nonblacks 

23    in terms of race and ethnicity.  There are no separate 

24    estimates for Hispanics, for Asian-Americans or Pacific 

25    Islanders or for other groups.
                                                              2080

 1         Q.    What, in your view, are the specific uses of 

 2    demographic analysis?

 3         A.    I would say there are three main specific uses of 

 4    demographic analysis.

 5         Q.    What is the first one?

 6         A.    The first is for providing estimates and bounds 

 7    on net national undercount and estimates on bounds on 

 8    differentials in undercounts, differentials between blacks 

 9    and nonblacks, between men and women, between different age 

10    groups.

11         Q.    What is the second main use?

12         A.    The second main use is for more detailed 

13    comparisons of undercounts for the groups for which 

14    demographic analysis provides undercount estimates:  That 

15    is, for groups by age, sex, and race in terms of black and 

16    nonblack.  We can compare the undercount estimates from the 

17    demographic analysis in detail to the undercount estimates 

18    from the PES.  Where they agree, they provide corroboration 

19    for the PES estimates.  Where they disagree, they point up 

20    discrepancies or anomalies that we then need to explain.

21         Q.    Can you give the Court an example of this?

22         A.    There is one big example.  That is the example of 

23    adult black males.  For most of the age groups of adult 

24    black males, the PES finds much lower undercount rates than 

25    the demographic analysis.  A finding of this kind is 
                                                              2081

 1    expected because most people expect that adult black males 

 2    are disproportionately likely to be missed by both the 

 3    census and the PES, and therefore omitted from the PES 

 4    estimates of undercount.  So that discrepancy, in terms of 

 5    its direction, is not surprising.  

 6               But what was very surprising when the PES 

 7    estimates and the demographic analysis estimates were first 

 8    compared was the magnitude of the discrepancies.  The 

 9    discrepancy is very large, and that was surprising, 

10    disturbing, and a lot of attention has been given to it both 

11    in the Secretary's report and ever since.

12         Q.    Does this phenomenon has a name?

13         A.    This kind of discrepancy gives rise to the bias 

14    we have called correlation bias.  The people involved are 

15    called unreached people.

16         Q.    I believe you mentioned a third main use of 

17    demographic analysis.  Could you tell the Court about that?

18         A.    That use really follows from the particular 

19    discrepancy I just mentioned.  The Census Bureau made 

20    attempts to use demographic analysis to gauge the extent of 

21    correlation bias or to gauge the numbers of unreached 

22    people.

23         Q.    What is the demographic analysis estimate of net 

24    national undercount?

25         A.    It is 1.85 percent.
                                                              2082

 1         Q.    In your opinion, is this estimate biased or 

 2    unbiased?

 3         A.    In terms of biases that we can identify or that 

 4    one might have been able to remove from the estimates, I 

 5    don't think there are identifiable biases.  I think the net 

 6    national undercount figure, to the extent of our present 

 7    knowledge, is unbiased, and I think the same is true for the 

 8    demographic analysis estimates of most of the groups.

 9         Q.    Dr. Wachter, is there uncertainty around these 

10    point estimates?

11         A.    Let me first mention that in saying the estimates 

12    are one biased, I am agreeing with Professor Estrada, who 

13    also testified to the Court that he found the estimates 

14    which he used in his report unbiased, to the best of his 

15    knowledge.  But both Professor Estrada and I emphasize the 

16    uncertainty in the estimates.  There is very substantial 

17    uncertainty in the estimates from demographic analysis.

18         Q.    Did the Census Bureau estimate uncertainty ranges 

19    or intervals around the point estimates?

20         A.    Yes the Census Bureau initiated a new program in 

21    the 1990 census program for putting intervals around its 

22    estimates from demographic analysis, uncertainty intervals.  

23    This program draws on all the D project evaluations, but the 

24    construction of the uncertainty ranges is found in the D-11 

25    project.
                                                              2083

 1         Q.    In your expert opinion, are the ranges of 

 2    uncertainty around the point estimates important?

 3         A.    Yes.  Yes, they are very important.

 4         Q.    Why is that?

 5         A.    What we are trying to learn from demographic 

 6    analysis is principally what possibilities are ruled out by 

 7    these numbers.  Any inferences we try to draw from the 

 8    demographic analysis numbers have to take these ranges of 

 9    uncertainty into account.  I would like to give you an 

10    example, actually.  

11               I was in court and heard Professor Fienberg 

12    testify and suggest that in terms of the differential 

13    undercount between blacks and nonblacks, the 1990 census 

14    was, quote, the worst in 50 years, the worst since 

15    demographic analysis estimates became available.  Professor 

16    Fienberg was referring, I believe, to the DA estimates that 

17    are found in Section 14.1 of the Secretary's report, which 

18    has given a point estimate of 4.2 percent for that 

19    differential.  

20               What Professor Fienberg didn't mention is that 

21    those estimates are subject to very wide uncertainty.  In 

22    fact, the width of the Bureau's uncertainty interval around 

23    that 1990 estimate is three whole percentage points.  It 

24    goes from 3.2 percent to 6.2 percent.  And the lower range 

25    of the Bureau's interval, the 3.2 percent, is actually lower 
                                                              2084

 1    than any of the other estimates from 1940 on.  

 2               So the real information is that we don't know 

 3    whether the differential between blacks and nonblacks in 

 4    1990 undercount is the smallest differential we have had in 

 5    50 years or the largest differential we have had in 50 

 6    years.  We know that there is a differential.  A 

 7    differential has persisted.  It hasn't gone away.  But we 

 8    don't know whether we have done better or we have done 

 9    worse.  

10               To use these intervals without taking the 

11    uncertainty ranges -- to use the estimates without taking 

12    the uncertainty ranges around them into account is to 

13    misrepresent the state of what we know.

14         Q.    You may have testified to this.  Where are the 

15    Bureau's ranges of uncertainty found?

16         A.    They are given in detail in the D-11 report.  

17    Perhaps I should mention that there was a 0.02 percent 

18    correction between the D-11 report and the Secretary's 

19    report.  I will usually refer to the D-11 figures, which are 

20    more complete.  Nothing depends on that tiny correction.  So 

21    they are given in detail in the D-11 report.  

22               A selection of them is given in Section 14.2, 

23    Tables 1 and 2, as I remember, in the Secretary's report.  

24    And the particular interval that I just gave you for 

25    Professor Fienberg's number I calculated myself from the 
                                                              2085

 1    information I had from the D-11 report.

 2         Q.    Do you accept the census Bureau's intervals?

 3         A.    No, I don't.  This was a new and somewhat 

 4    experimental program.  There were two stages in the Bureau's 

 5    assessment of uncertainty.  One was the assessment of 

 6    uncertainty in the components, the basic building blocks of 

 7    the demographic analysis.  I do accept that detailed 

 8    demographic work.  I believe the Bureau came up with 

 9    realistic assessments in the components, in the basic 

10    building blocks.  

11               The second stage of the Bureau's work, detailed 

12    in D-11, involve formulas for combining those intervals when 

13    you go up to larger groups.  That second stage contains 

14    definite flaws.

15         Q.    Have you investigated the effects to be expected 

16    from removing these flaws?

17         A.    I have.  I have investigated this question at 

18    considerable length, and there are two levels, two kinds of 

19    flaws in the intervals.  I have investigated what the 

20    effects of removing each of them might be.

21         Q.    Let me return for a moment, Professor Wachter --

22         A.    Let me say, since you have asked that question, 

23    the main effects from the point of view of the numbers we 

24    were just discussing, the net differential for 1990, the 

25    main effect is that the Bureau's intervals are actually 
                                                              2086

 1    inconsistently narrow.  They are narrower than would be 

 2    compatible with the basic demographic judgments that went 

 3    into them, and they also have an incorrect balance of 

 4    uncertainty allocated to the low side of the estimate and 

 5    the high side of the estimate.  

 6               So corrected intervals would lead to even greater 

 7    uncertainty than the uncertainty that I described in the 

 8    Bureau's intervals, both in this case and in most cases.

 9         Q.    Let me ask you about the second main use you 

10    mentioned for demographic analysis.  Am I correct that these 

11    were comparisons between DA estimates of undercounts and PES 

12    estimates of undercounts for specific age/sex/race groups?

13         A.    Yes, for detailed comparisons.

14         Q.    Did the Secretary use demographic analysis for 

15    such specific comparisons?

16         A.    Yes, he did.  There is an extended passage, I 

17    think it is pages 2-10 to 2-12 in the Secretary's report, in 

18    which he uses the demographic analysis estimates of 

19    undercount and the uncertainty intervals along with the PES 

20    estimates and uncertainty intervals to look for match and 

21    mismatch.

22         Q.    Do you consider that an appropriate use of 

23    demographic analysis?

24         A.    I do.  That is one of the main purposes of doing 

25    such a detailed program of demographic analysis.
                                                              2087

 1         Q.    Could I refer your attention to what has been 

 2    marked as Defendants' Exhibit 48.  I believe that is in the 

 3    second binder of defendants' exhibits.  

 4               MR. BARON:  This, your Honor, is an excerpt of 

 5    the administrative record which is already in evidence.  For 

 6    the convenience of the Court, we have separated it out.

 7         Q.    Would you turn to pages 5 and 6, Professor 

 8    Wachter.  Do you see the paragraph at the bottom of page 5, 

 9    which starts with "This property of the demographic 

10    estimates"?  If you could read that paragraph and the 

11    continuing paragraph to the end of page 6.

12         A.    Yes.

13         Q.    Do you recall being in the courtroom when 

14    Professor Estrada discussed sections of this text?

15         A.    Yes, these sections of the text.

16         Q.    Do you agree or disagree with what Professor 

17    Estrada testified to?

18         A.    I don't believe that Professor Estrada was 

19    drawing the right conclusion from this passage.  This is a 

20    memo by Dr. Robinson and Dr. Das Gupta who were responsible 

21    for the uncertainty intervals.  

22               This passage is pointing out properties of the 

23    Bureau's uncertainty intervals.  Some of the intervals are 

24    narrow and some of the intervals are wider.  Some of the 

25    intervals for differentials are narrower than some of the 
                                                              2088

 1    intervals for groups.  This passage is giving reasons why, 

 2    in Dr. Das Gupta's and Dr. Robinson's view, the narrower 

 3    intervals for differentials are not implausible on the face 

 4    of it.  

 5               I actually think there are some reasons to 

 6    believe that those narrower intervals are implausible on the 

 7    face of it, but here they are making a kind of general case 

 8    that the intervals are not implausible on the face of it.  

 9               But they are not cautioning against the use of 

10    the intervals.  They are describing what the intervals are.  

11    They are not telling the Secretary not to trust the 

12    intervals for groups; they are pointing out that the 

13    intervals for groups differ from the intervals for 

14    differentials.

15               MR. BARON:  Let me just note for the Court and 

16    for the convenience of opposing counsel that Professor 

17    Estrada's reference to this is on page 889 of the transcript 

18    starting at lines 14, that sequence in I believe it was 

19    Plaintiffs' 4 which we referred to, which is the same as 

20    Defendants' 48.

21         q.    Professor Wachter, in your view, does the 

22    Secretary's report show a misuse of the information 

23    presented in Defendants' Exhibit 48?

24         A.    No.  The Secretary is using the estimates and 

25    uncertainty intervals for exactly the purpose for which they 
                                                              2089

 1    were constructed.  We have ranges of uncertainty, and those 

 2    permit us to ask whether a given difference between 

 3    demographic analysis undercount estimates and post- 

 4    enumeration survey estimates is or is not greater than can 

 5    be explained by the uncertainties in the two sets of 

 6    estimates.  

 7               What the Secretary is doing is following up on 

 8    the whole purpose of the census Bureau's project.

 9         Q.    The Secretary doesn't discuss broad differentials 

10    and undercounts in his portion of the report?

11         A.    No.  He is looking here at specific comparisons.  

12    The Secretary acknowledged the existence of broad 

13    differentials in undercounts.  That is the starting point of 

14    his report.  Somehow, the whole decision document is 

15    predicated on that.  

16               But what the guidelines required was a judgment 

17    about the relative accuracy of adjusted and unadjusted 

18    counts.  For that purpose it is important to use the 

19    information from demographic analysis to the extent it sheds 

20    light on what we see in the PES.  And that is what he was 

21    doing here.

22         Q.    Professor Wachter, I note that you have 

23    Plaintiffs' Exhibit 9, which is the Secretary's decision, 

24    with you.  

25               MR. BARON:  Does the Court have a copy of 
                                                              2090

 1    Plaintiffs' Exhibit 9?  

 2               THE COURT:  Yes.

 3         Q.    Turn to page 2-10 and 2-12.

 4         A.    Yes.

 5         Q.    What, in your opinion, Professor Wachter, does 

 6    the discussion on these pages 2-10 through 2-12 of the 

 7    Secretary's report show in terms of comparisons being made?

 8         A.    The discussion shows that something is wrong.  

 9    There are quite a number of cases in which the differences 

10    between the demographic analysis estimates and the PES 

11    estimates are outside the ranges of uncertainty.  So 

12    something has to be wrong.

13         Q.    Tell the Court what you believe could be wrong.

14         A.    There are at least three main things that could 

15    be wrong.  First, the uncertainty intervals for the post- 

16    enumeration survey estimates could be wrong.  They could, 

17    perhaps, inadequately take account of the systematic biases 

18    or nonsampling errors actually in the PES.  

19               A second possibility is that the uncertainty 

20    intervals for the post-enumeration survey estimates could be 

21    wrong because they inadequately take account of the random 

22    or sampling variability in the PES.  

23               The third possibility is that the uncertainty 

24    intervals for the demographic analysis could be wrong.  The 

25    errors in the demographic analysis are almost entirely 
                                                              2091

 1    nonsampling errors.  So this is to say that the treatment of 

 2    nonsampling errors in the uncertainty intervals could be 

 3    wrong.

 4         Q.    Do any of these possible answers, in your 

 5    opinion, turn out to be true?

 6         A.    From what I have seen, all three turn out to be 

 7    true.  The systematic errors or biases in the PES are not 

 8    adequately represented in the intervals for the PES.  

 9    Indeed, the Secretary alludes to this point actually on page 

10    2-10.  

11               I have seen reports by Dr. Breiman, very detailed 

12    and careful analyses, which suggest -- 

13               MR. ZIMROTH:  Your Honor, I'm sorry, these were 

14    excluded.  I don't think this witness ought to be allowed to 

15    testify about what your Honor just excluded.  

16               MR. BARON:  If it forms the basis of his expert 

17    opinion, whether or not it has been included from the 

18    evidence.  

19               THE COURT:  I will permit it.

20         A.    I am not going to go into the testimony that 

21    Professor Breiman might have made.  

22               THE COURT:  Just advert to it.

23         A.    But I am convinced from my study of Dr. Breiman's 

24    report that there are substantial additional measured biases 

25    in the PES which are not included in the Bureau's intervals, 
                                                              2092

 1    so I am convinced that the first thing that could be wrong 

 2    here is wrong.  

 3               The second point is the sampling errors.  We 

 4    heard Dr. Fay testify that the estimates of variance, which 

 5    are the measures of sampling errors in the PES, are 

 6    understated, possibly very substantially understated, which 

 7    means that the uncertainty intervals for the PES estimates 

 8    that go into this comparison of the Secretary's are 

 9    understated.  So this second thing that could go wrong did 

10    go wrong.  

11               Then we come to the third thing, the uncertainty 

12    intervals for demographic analysis.  This was the part of 

13    the puzzle that I set out to analyze myself, reflecting on 

14    these examples of the Secretary's report.  As I mentioned, I 

15    found two definite flaws in the uncertainty intervals, and I 

16    now believe the demographic uncertainty intervals should be 

17    substantially broader and somewhat differently balanced 

18    between low-side and high-side uncertainty.  

19               So this is one of these cases where there were 

20    clues in the Secretary's report, and when you follow up on 

21    those clues, you find all three of the possible explanations 

22    panning out.

23         Q.    Did you point out such anomalies in your own 

24    recommendation to the Secretary?

25         A.    Not to any extent.  I make some mention of the 
                                                              2093

 1    issue.  But my report was sent on the 17th of June, and the 

 2    Secretary's decision was on the 15th of July, so my report 

 3    is a whole month earlier than the Secretary's.  I didn't 

 4    have sufficiently comparable numbers for the PES and the 

 5    demographic analysis to make much in the way of detailed 

 6    comparisons.  

 7               There were two main problems.  I didn't have 

 8    counts for the institutional population.  The institutional 

 9    population is left out of the PES but included in the 

10    demographic analysis.  I also didn't have a good basis for 

11    identifying blacks in the PES.  

12               What I had was blacks in the black poststrata, 

13    the poststrata that are assigned to blacks.  But there are 

14    some blacks who are in poststrata that combine blacks and 

15    Hispanics or combine other groups.  I didn't have 

16    information to separate out those numbers.  

17               So while I offered some suggestive comparisons, I 

18    pointed out in my report that better numbers would be 

19    available to the Secretary.  Of course, those better numbers 

20    were available to the Secretary.  He didn't rely on my 

21    numbers.

22         Q.    On the whole, Professor Wachter, do you consider 

23    the concerns about anomalies expressed on pages 2-10 through 

24    2-12 of the Secretary's report to be rational?

25         A.    Yes.  They are entirely rational.  This is what 
                                                              2094

 1    the DA estimates were constructed for, and in this case 

 2    these analyses point us toward what I consider important 

 3    further scientific discoveries about the PES and the 

 4    demographic analysis.

 5         Q.    Professor Wachter, are all the discrepancies 

 6    which the Secretary noted between DA and PES explained away 

 7    by the three sets of findings you have just mentioned?

 8         A.    Most of the specific discrepancies are explained 

 9    away, the discrepancies for particular age or sex groups.  

10    But the main discrepancy that I described, the discrepancy 

11    for adult black males, is not explained away.  On my 

12    somewhat detailed study I don't believe you can explain it 

13    away with reference to uncertainties in the different 

14    components.  It's there.

15         Q.    Could you now explain for the Court precisely 

16    what the term "unreached people" means?

17         A.    Yes.  This is a term which I actually coined.  It 

18    refers to a subset of all the people who are missed by both 

19    the census and the PES.  That subset of people are people 

20    who are missed in addition to an allowance for such people 

21    in the dual-system estimates.  

22               So the capture-recapture method, the dual-system 

23    estimates, make an allowance for people who are missed in 

24    both the census and the PES.  They make it using a certain 

25    homogeneity assumption.  And there are about 1.5 million 
                                                              2095

 1    people included in the adjusted counts who are missed in 

 2    both the census and the PES.  But unreached people are 

 3    people who may have been in the census -- sorry -- may have 

 4    been in the population but missed by both the census and the 

 5    PES in excess of that allowance.

 6         Q.    Am I correct that earlier you listed as a use of 

 7    demographic analysis the Bureau's attempts to gauge the 

 8    numbers of such unreached people?

 9         A.    Yes.  They made some fascinating and bold 

10    experimental attempts to gauge these numbers.  The most 

11    important of these attempts are based on sex ratios, ratios 

12    of the number of males to females in age groups and race 

13    groups.  Those sex ratios with estimates of uncertainty were 

14    presented in the P-17 project, and they were then taken over 

15    for the P-13 project, which attempted to use them to gauge 

16    possible numbers of unreached people.  

17               (Continued on next page) 

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2096

 1         Q.    Are you saying that demographic analysis can be 

 2    used to measure correlation bias?

 3         A.    Well, measure is much too strong a word.

 4               It gives us a pretty good idea that there are 

 5    unreached people, but it doesn't really give us measurements 

 6    of the numbers, and there are two reasons for this:

 7               First of all, the phenomenon of correlation bias 

 8    really enters into the PES at the level of post-strata and 

 9    local areas.  That's where the problem is a problem, and the 

10    demographic analysis doesn't tell us anything about the 

11    national level, so there is no measurement from demographic 

12    analysis of local level correlation bias.

13               Even at the national level, the uncertainties in 
                                                                  

14    the demographic analysis and the problems and uncertainties 

15    with the sex ratios are great enough that we don't, we don't 

16    really get a measurement, we get an idea, we get a general 

17    idea.

18         Q.    Tell the court why the quote unreached people 

19    unquote are important in the whole scheme of thing?

20         A.    The unreached people are very important from the 

21    point of view of whether the adjusted counts give better 

22    distributive accuracy than the unadjusted counts.  They are 

23    a source of potential error in the unadjusted counts which 

24    we have practically no handle on, no way to measure, no 

25    clear idea of how they are distributed, and so they are the 
                                                              2097

 1    wild card in this whole analysis.

 2         Q.    Are they important in terms of calculating 

 3    adjusted shares for states and local areas?

 4         A.    Yes, that's their, that's their chief importance.

 5               We, we don't believe that the unreached people 

 6    are negative people, we believe the correlation bias is 

 7    positive, so these unreached people constitute an unmeasured 

 8    bias in the post-enumeration survey which goes in a 

 9    predictable direction.

10               But when we are considering shares, if there are 

11    more unreached people in one area and fewer in another, the 

12    effect of the unreached people on the accuracy of the shares 

13    can't be predicted directly, so the unreached people have a 

14    potentially large impact on the distributive accuracy of 

15    adjusted counts.

16         Q.    Tell the court again which studies the Bureau 

17    performed which gauged numbers of unreached people?

18         A.    The main study is based on the demographic 

19    analysis sex ratios which are presented and discussed in 

20    P17 and then those data are used for the specific purpose in 

21    the P13 project.

22         Q.    You were in court when Professor Estrada 

23    testified, correct?

24         A.    Yes.

25         Q.    Do you recall the following exchange which is at 
                                                              2098

 1    page 911 of the transcript of these proceedings starting at 

 2    line 11, page 911, line 21.

 3               "Q.    Was it your view at the time that I took 

 4    your deposition that the information in P17 had always been 

 5    problematic to you?

 6               "A.    The P17 figures have been problematic to 

 7    me, yes." 

 8               Do you recall hearing that?

 9         A.    I do.

10         Q.    Do you agree with Professor Estrada?

11         A.    Yes, I agree with Professor Estrada.

12               If I can give a couple of reasons why they are 

13    problematic, if you like.

14         Q.    Go right ahead.

15         A.    Or I can just agree with him.

16               Well, when you use the sex ratios for P17 in the 

17    way the Bureau used them, you get very irregular patterns by 

18    age for the putative numbers of unreached people.  They are  

19    very irregular patterns that look internally inconsistent.

20               There are also uncertainty intervals for the 

21    seconds ratios quoted in P17 which are, I think, pretty 

22    clearly not right.  Those uncertainty intervals leave out 

23    the main source in the uncertainty, one of the main sources, 

24    I would say the main source in the uncertainty that's built 

25    into the demographic analysis estimates, and this fact is 
                                                              2099

 1    pointed out in the P17 project, but pointing it out doesn't 

 2    solve the problem, the information is problematic and of 

 3    somewhat limited usability.

 4         Q.    Thank you.

 5               Is there a report on demographic analysis which 

 6    you have prepared?

 7         A.    Yes.

 8         Q.    I turn your attention, Professor Wachter, to what 

 9    has been marked as Exhibit 40 in the volume of defendants' 

10    exhibits, and I ask you, can you identify this document?

11         A.    This includes both a report and a more technical 

12    appendix which was written by me and which presents the 

13    results of my study of the Bureau's demographic analysis and 

14    of the flaws in the uncertainty intervals.  

15               MR. BARON:  May I approach, your Honor? 

16               (Handing to the court)

17               (Pause)

18         Q.    Are your reasons for treating the information in 

19    P17 as used in P13 a subject set out in detail in this 

20    report?

21         A.    Yes, they are.

22         Q.    Professor Wachter, you testified earlier, did you 

23    not, that you believe that the Bureau's uncertainty 

24    intervals for estimates from demographic analysis were 

25    flawed.
                                                              2100

 1               Were you referring solely to the intervals in P17 

 2    as used in P13?

 3         A.    No.  I believe the general procedure that the 

 4    Bureau used at the last stage for combining the basic 

 5    estimates and intervals into estimates and intervals for 

 6    larger groups, like race, sex, age group and particularly 

 7    race and sex groups of national population, the flaws I'm 

 8    referring to affect all those intervals.

 9         Q.    And what affect do these flaws have?

10         A.    They have two main effects:

11               First of all, they lead to uncertainty intervals 

12    for large groups which are spuriously lopsided and they also 

13    lead to intervals for large groups which are unrealisticly 

14    narrow.

15         Q.    Can you explain to the court what you mean by 

16    spuriously lopsided?

17         A.    Sure.

18               Let's think about the interval for net national 

19    undercount, the demographic estimate in the D11 project for 

20    net national undercount is 1.83 percent and the Bureau's 

21    interval stretches from 1.63 percent up to, if I remember 

22    correctly, 3.34 percent, approximately 3.3 percent, so you 

23    see there is only zero .2 percent difference between the 

24    bottom of the interval, that 1.63, and the Bureau's best 

25    estimate, the 1.83, whereas there is something like seven or 
                                                              2101

 1    eight times as much uncertainty on the high side stretching 

 2    all the way from 1.83 percent to 3.34 percent.

 3               Now, I say that it's spuriously lopsided because 

 4    that kind of difference between our uncertainty on the low 

 5    side and our uncertainty on the high side isn't actually 

 6    found in the basic demographic judgments that went into the 

 7    Bureau's intervals, that is, the judgments for the 

 8    components, the small building blocks.  Those show a modest, 

 9    a modestly greater uncertainty on the high side than the low 

10    side, approximately a 55/45 percent split.  It's very 

11    modest.

12               But these flaws in the combination procedure 

13    turned that modest imbalance into this dramatic imbalance.  

14    The technical term for this is skewing, and in my report I 

15    say that these flaws into the combination procedure 

16    introduced extraneous skewing.  That's exactly what they 

17    did.

18         Q.    Can the effects of what you say is extraneous 

19    skewing be corrected?

20         A.    There is a fairly simple correction that can be 

21    made which would bring the Bureau's intervals for larger 

22    groups into line with the Bureau's basic information for the 

23    components for the basic building blocks so as to make the 

24    Bureau's intervals, the skewing in the Bureau's intervals 

25    consistent with the basic judgments.  That's a fairly 
                                                              2102

 1    straightforward correction.

 2               When you do that, you get something like 45 

 3    percent of the uncertainty on the low side and 55 percent of 

 4    the uncertainty on the high side, so you got an interval 

 5    that is now starting at something like 1.09 percent and 

 6    going up to something like 2.8 percent.

 7         Q.    Well, this is pretty technical discussion which 

 8    we have been having, but what does that correction in terms 

 9    of correcting the skewness of the intervals imply for net 

10    national undercount?

11         A.    Well, it implies that we really cannot rule out 

12    figures down near one percent for net national undercount, 

13    and especially when you combine this with corrections or 

14    possible corrections for other flaws in the analysis, the 

15    flaws in the Bureau's analysis, the flaws that make the 

16    intervals also unrealisticly narrow, you end up with 

17    intervals that are stretching down on the low side to 

18    something like zero .7 percent and stretching up on the high 

19    side to 3.4 or 3.5 percent.

20               So the revised intervals give you a very 

21    different picture of what kind of information the 

22    demographic analysis is giving you.

23               Let me spell this out in terms of the point that 

24    is probably most relevant to the court.

25               When the Bureau considered the nonsampling 
                                                              2103

 1    errors, the measured biases in the post-enumeration survey 

 2    in the P16 project, it found that the measured biases on 

 3    balance tended to make the PES estimates appear as 

 4    overstatements, so the correction for measured bias would 

 5    bring the PES estimates of undercount down.

 6               Now, if we believed the Bureau's original 

 7    uncertainty intervals and we believed that demographic 

 8    analysis really ruled out undercounts below something like 

 9    1.63 percent, then the more allowance for measured bias you 

10    make to the PES estimates, the more you are forced to assume 

11    there must be unmeasured biases that we haven't measured 

12    presumably in the form of more and more unreached people.

13               So if you have the 1.63 percent lower bound that 

14    the Bureau had in July, then all our analysis of measured 

15    biases force on us ever more unmeasured bias.

16               Once you recognize that those intervals were 

17    spuriously lopsided and unrealisticly narrowed, you see that 

18    we are not forced on that.  If the lower bound is more like 

19    zero .7 percent, then you could have a substantial amount of 

20    measured bias in the PES without having huge amounts of 

21    additional unmeasured bias forced upon you.

22               So this result really opens up the possibility 

23    that you can have a story about unmeasured bias that is not 

24    driven by your story of measured bias, and this is really 

25    the issue that we are grappling with in the whole question 
                                                              2104

 1    of whether the adjusted or unadjusted counts are more 

 2    accurate.

 3         Q.    Are the details of this analysis set out and 

 4    explained in your report?

 5         A.    The details of my analysis of the uncertainty 

 6    intervals and the demographic estimates are set out in my 

 7    report.  These include fairly technical issues and those are 

 8    treated in the report.

 9         Q.    Did the results from demographic analysis rule 

10    out the possibility that the census count is more accurate 

11    in numeric terms than the adjusted count at the national 

12    level?

13         A.    No, they don't rule it out.  That lower bound, 

14    the zero .7 percent is actually closer to the census 

15    undercount figure, the census undercount figure is zero 

16    percent, than it is to the PES figure, which is, which is, 

17    from the point of view of these estimates, is 2.11 percent, 

18    so the demographic analysis doesn't actually rule out the 

19    possibility that the census had better numeric accuracy at 

20    the national level than the adjustment.

21               But the interval, on the other side, goes up to 

22    3.34 percent, so I'm not saying that the Secretary was wrong 

23    to believe it was very likely that the adjusted counts had 

24    greater numeric accuracy at the national level than the 

25    census counts, I think the Secretary was accepting the most 
                                                              2105

 1    likely possibility and I accept it, too, but the demographic 

 2    analysis is a very fuzzy yardstick for making the comparison 

 3    and it doesn't actually rule out the opposite possibility.

 4         Q.    Is there any other general conclusion you wish to 

 5    make with respect to your testimony on demographic analysis?

 6         A.    I think I can sum up what I've been saying in the 

 7    following way:

 8               It would be wonderful if we had an independent 

 9    yardstick for assessing the relative accuracy of adjusted 

10    and unadjusted counts and at the national level the 

11    demographic analysis is the only thing out there that we 

12    have which might provide such a yardstick.

13               But when we study carefully the Bureau's work in 

14    the D projects, we see that the uncertainties in demographic 

15    analysis are just too great to give much confirmation or 

16    really to point up much in the way of differences between 

17    them.  The yardstick is just too fuzzy.

18               So I'm basically aligning myself with the 

19    testimony Dr. Estrada gave.  Dr. Estrada said the 

20    uncertainties in demographic analysis are too great to use 

21    for firm comparison with the PES, and that's also my 

22    conclusion.  

23               MR. BARON:  At this time I move the admission of 

24    Defendant's Exhibit 40 into evidence.  

25               MR. GOLDIN:  Your Honor, I would object.
                                                              2106

 1               This is a rather lengthy report which purports to 

 2    be a report by Dr. Wachter.  It is an unsworn out of court 

 3    statement, it is one of about a half dozen that we 

 4    anticipate from defendants' representations will be offered 

 5    through this witness.

 6               I understand that previously the court has 

 7    received into evidence work being done at the Bureau 

 8    subsequent to July 15, 1991, but I do not see the relevance 

 9    of work that is being done by somebody not at the Bureau, 

10    not working with the Bureau, not working with the Bureau's 

11    data, do not bear upon the material that was before the 

12    Secretary or the rationale for his decision nor can it shed 

13    light on work that is currently being done by the Bureau in 

14    connection with the PES.

15               If these documents, this report and others are 

16    received through Dr. Wachter, we have imposed upon 

17    plaintiffs the burden of cross-examining him on a great mass 

18    of material that will be coming before the court without 

19    this witness testifying about it on direct, essentially be 

20    deposited with the court with only a brief introduction by 

21    the witness.

22               On that basis, I think that it is inappropriate 

23    to receive it and we would object.  

24               THE COURT:  Mr. Baron.  

25               MR. BARON:  Your Honor, this is a report prepared 
                                                              2107

 1    by one of defendants' expert witnesses.  It is a report much 

 2    like any other report an expert would prepare to form the 

 3    basis of the expert's opinion.

 4               The report, this one and others that I will be 

 5    introducing today, were proffered to plaintiffs' counsel as 

 6    part of the exchange which was ordered on May 1 by prior 

 7    order of this court.  This court set out a procedure in this 

 8    action by which there would be depositions of each other's 

 9    experts and there would be a proving up of reports of those 

10    experts by May 1.

11               We have done that.  Mr. Goldin has had since May 

12    1 to have this report in his hands, do what he wishes with 

13    it.  I think it's appropriate in that it adds measurably to 

14    the record, because it is in detail what has been testified 

15    to with respect to demographic analysis and there are other 

16    reports of the same type that we have, and I would ask that 

17    it be received into evidence at this time.  

18               THE COURT:  What does it add to his testimony?  

19               MR. BARON:  It provides in greater detail what 

20    findings and conclusions this expert has with respect to 

21    demographic analysis.

22               We certainly would not --  I did not offer 

23    Professor Wachter coming here today to go through page by 

24    page of this report.  That's not what I burden the court 

25    with this morning.
                                                              2108

 1               If the report was not to be entered into 

 2    evidence, I would have constructed a much more detailed set 

 3    of questions which would get in in very great detail what 

 4    the various findings and conclusions were in the report.  I 

 5    didn't want to burden the court with that.  However, we have 

 6    given this up weeks ago and Mr. Goldin certainly is capable 

 7    of cross-examining this witness on the basis of this report 

 8    as well as his testimony this morning.  

 9               MR. GOLDIN:  Your Honor, if I might just respond 

10    briefly to something that Mr. Baron said.

11               It is true that the defendants served upon us 

12    copies of their numerous reports on May 1.  That was not 

13    pursuant to an agreement or a ruling that specifically 

14    pertained to experts' reports, that was simply the result of 

15    the arrangement under which the parties were to serve upon 

16    one another copies of any exhibits that were to be 

17    introduced on May 1.

18               Plaintiffs chose to obtain those reports and 

19    cover them under that rubric.

20               Of course, the depositions of the pertinent 

21    witness occurred before May 1 so there was no opportunity to 

22    depose any of the witnesses who were submitting reports on 

23    the subject of the reports, and with respect to all of the 

24    material that the defendants' experts have been included in 

25    their reports and testifying about defendants have taken the 
                                                              2109

 1    position that they are obligated to reveal underlying data 

 2    only the day before the witness testifies, with certain 

 3    exceptions, one of which occurs in the case of Professor 

 4    Wachter, which I imagine will come in due course, where we 

 5    have not obtained the underlying data even today.

 6               In any event, your Honor, it seems to me that 

 7    whatever Mr. Barron's concern about not burdening the court, 

 8    the burden is placed on plaintiffs of having this mass of 

 9    material insinuated into the record.  Our obligation is not, 

10    I believe, to respond to it.  Defendants have simply 

11    relieved themselves of the opportunity of putting it before 

12    the court in the form of competent testimony.

13               We continue the objection.  

14               MR. BARON:  Your Honor, I have not been present 

15    for every minute of this trial.  However, I am informed from 

16    my co-counsel that, for example, Professor Cain testified as 

17    to work which he had performed in the time period up to the 

18    time of this trial and it's been received into evidence 

19    without objection by defendants.

20               I would submit to the court that this report and 

21    any other report like it where Professor Wachter is 

22    equivalent to the evidence which has been received from 

23    Professor Cain.

24               At the time of the depositions there were no 

25    objections that we had to a witness saying on the record at 
                                                              2110

 1    the time that he is working on further work or analysis and 

 2    that it would be produced at the time of trial.

 3               In the case of Professor Cain that is one example 

 4    where such evidence has been put before the court.

 5               I would say also that the court expressly ruled 

 6    in its March 24 conference call, and there was a motion to 

 7    reconsider on this which was denied, that expert documents 

 8    would be produced at the time of the May 1 exchange rather 

 9    than before the depositions in this action, and I would 

10    simply submit that Mr. Goldin's statements about the burdens 

11    on plaintiffs have no basis in the rules of evidence.  

12               MR. GOLDIN:  Your Honor, if I might --  

13               THE COURT:  I've heard enough.

14               I am going to overrule the objection and admit 

15    the document marked Defendant's Exhibit 40.  I believe it is 

16    admissible under Rule 703 as the basis of the expert's 

17    opinion which has been given in court. 

18               MR. BARON:  Thank you, your Honor.

19               (Defendant's Exhibit 40 marked for identification 

20    was received in evidence.) 

21    BY MR. BARON:

22         Q.    Professor Wachter, I want to talk to the 

23    assumption of the homogeneity assumptions.

24               Have you studied the assumptions called 

25    homogeneity assumptions in the 1990 census?
                                                              2111

 1         A.    Yes, I have.

 2         Q.    Will you tell the court what are these 

 3    assumptions?

 4         A.    The homogeneity assumptions are assumptions that 

 5    the probability of being missed or miscounted in the census 

 6    are the same for all the people who are in the same one of 

 7    the Bureau's 1,392 post-strata.

 8         Q.    What role did these assumptions play?

 9         A.    Well, the assumptions play several roles.

10               The first and most important role is that 

11    homogeneity assumptions are the key to the Bureau's method 

12    of synthetic estimation.  

13               This is the process by which adjustments are 

14    carried down from the 1,392 post-strata down to local areas, 

15    blocks and then back up to counties, cities, states and 

16    other local jurisdictions, and that synthetic adjustment is 

17    based on homogeneity assumptions.

18         Q.    Have you heard any other witness in this action 

19    call this synthetic process by another name?

20         A.    Yes.  I believe at the beginning of the trial 

21    Professor Ericksen called it the trickle-down process.  We 

22    are talking about the trickle-down process or the carry-down 

23    process.

24         Q.    Are homogeneity assumptions need at any other 

25    stage?
                                                              2112

 1         A.    A special case of homogeneity assumption is 

 2    needed in the capture-recapture formula in the preparation 

 3    of the dual system estimates.  That special case is the 

 4    assumption that the probability of being counted in the 

 5    census is the same for people in the same post-stratum 

 6    whether or not you are counted in the PES and the failure of 

 7    that assumption is what gives rise to the correlation bias 

 8    we have been talking about, so that is a homogeneity 

 9    assumption, also.

10         Q.    First let me ask you about a homogeneity 

11    assumptions in the synthetic process.  

12               THE COURT:  Let's take a five minute break at 

13    this point.  

14               MR. BARON:  Yes, your Honor.

15               (Recess) 

16               THE COURT:  All set? 

17               Mr. Baron.  

18    BY MR. BARON:

19         Q.    Where we broke off, I was going to ask you about 

20    homogeneity assumptions in the synthetic process.

21               Let me ask, is heterogeneity what is present when 

22    the homogeneity assumptions fail?

23         A.    Yes, that's right.

24         Q.    Okay.

25               When you speak of failure of the homogeneity 
                                                              2113

 1    assumption, do you mean the failure of the assumption to 

 2    hold exactly?

 3         A.    No, of course not.  We mean, is there enough 

 4    heterogeneity there left to cause trouble.

 5               The point has been made to the court before, we 

 6    start with a lot of heterogeneity all through America and 

 7    the Bureau tries to control this heterogeneity with its 

 8    post-stratification scheme breaking people down into 1,392 

 9    groups, so some of the heterogeneity is controlled and some 

10    of it remains in there.  That's the residual heterogeneity. 

11               And when we talk about the homogeneity assumption 

12    failing, we mean, is there much heterogeneity left in the 

13    post-strata, are people in the same post-stratum so 

14    different that significant extra errors are introduced into 

15    adjusted counts.

16         Q.    In your view, when the Secretary's report 

17    discusses failure of the homogeneity assumption, does the 

18    Secretary mean failure of the assumption to hold exactly?

19         A.    No, upon my reading of --  

20               MR. GOLDIN:  Objection, your Honor.  It calls for 

21    speculation.  

22               MR. BARON:  I asked him in his view.  

23               THE COURT:  Overruled.

24               You may answer.

25         A.    On my reading of the report, the report is 
                                                              2114

 1    discussing exactly the question that I have just discussed.

 2         Q.    Professor Wachter, were you present in the 

 3    courtroom when Dr. Feinberg stated the following.  I believe 

 4    this is at page 1498, trascript lines 18 to 25.

 5               Professor Fienberg stated this, "Look, some 

 6    assumptions we know don't matter very much, they are there 

 7    for technical convenience.  There are other ones that 

 8    sometimes you think may matter more and those you look at 

 9    and you check some sensitivity of the results to some 

10    clearly sensible alternatives in the context of what you 

11    know about the problem that you are dealing with and what it 

12    is you want to measure and how you can gauge that impact." 

13               Professor Wachter, is the homogeneity assumption 

14    one of those assumptions that quote don't matter very much 

15    unquote or which are quote there for technical convenience 

16    unquote only to paraphrase Dr. Feinberg?

17         A.    No.  The homogeneity assumption, in a sense, is 

18    the whole ballgame.  It's in the synthetic process that the 

19    use being made of the PES differs from the use we make of 

20    virtually all other national surveys.

21               In most surveys we try to generalize up from a 

22    sample to a whole population, but in the post-enumeration 

23    survey, we're not only generalizing up from a sample to the 

24    whole population, but then we are trying to generalize down 

25    to local areas, to 39,000 jurisdictions, ultimately to 
                                                              2115

 1    upwards of 4.5 million occupied blocks, and it's this 

 2    process of generalizing down that isn't generally done in 

 3    survey applications, certainly not on this scale and with 

 4    this specificity.

 5               So whether this works or not, whether we 

 6    introduce large amounts of extra error from this process is, 

 7    in a sense, the main question about the PES.  

 8               This is the first time something like this has 

 9    been tried on this kind of scale with sufficient evaluation 

10    studies and sufficient care to evaluate it and whether this 

11    works or not is really the big open question about 1990 

12    census adjustment.  

13               MR. GOLDIN:  Your Honor, I did not want to 

14    interrupt the witness, but I would note for the record that 

15    when counsel referred to a question and answer involving the 

16    testimony of Dr. Feinberg, in setting up that question, 

17    there was, in fact, in Dr. Fienberg's testimony, nothing 

18    referring to the homogeneity assumption, contrary to the 

19    implication in counsel's question to this witness.  

20               MR. BARON:  Your Honor, I did not imply that or I 

21    did not mean to imply it, I was talking about something in 

22    general.  

23               THE COURT:  I did not think you did, either.  

24               MR. BARON:  Thank you, your Honor.  

25               MR. BARON:  Let me put this up on the chart, 
                                                              2116

 1    which is marked as Defendant's Exhibit 41 A.  

 2               It is, for the court, a chart which is a colored 

 3    version of a chart which appears in the report that is at 

 4    Defendant's Exhibit 41.  The chart in the report appears as 

 5    figure 1 after page 4 in Defendant's Exhibit 412 in the 

 6    binder.  

 7    BY MR. BARON:

 8         Q.    Professor Wachter, do you recognize this diagram?

 9         A.    I do.  

10               MR. GOLDIN:  I'm sorry, your Honor, I have to 

11    object, again.

12               We did obtain a copy of this report from 

13    defendants on May 1.  The version of this chart which 

14    appears there is a black and white reproduction --  

15               THE COURT:  Appears there, where do you mean?  

16               MR. GOLDIN:  In the copy of the report that was 

17    provided to us on May 1.  

18               THE COURT:  It is the same as figure 1 as 

19    attached to Defendant's Exhibit 41?  

20               MR. GOLDIN:  That's correct, your Honor.

21               But by virtue of the black and white 

22    reproduction, it is significantly different from the chart 

23    which is being introduced here.  

24               As I think a visual inspection of the document 

25    before your Honor will reveal, it's very difficult to assume 
                                                              2117

 1    from that document before your Honor that the chart is going 

 2    to appear as we see it in the marked exhibit.

 3               On that basis, I would object that we have not 

 4    been provided with this exhibit as was supposed to be done, 

 5    and I would object to its introduction into evidence or its 

 6    use for illustrative purposes.  

 7               MR. BARON:  Well, gosh --  

 8               THE COURT:  It's the same exhibit, except that 

 9    this one is in full chromaticism.  

10               MR. GOLDIN:  Well, I recognize that, your Honor.  

11               THE COURT:  Other than that, I don't have any 

12    trouble with it.

13               There is no jury here.  I won't be mislead by it.  

14    The objection is overruled.  

15               MR. GOLDIN:  Thank you, your Honor.  

16    BY MR. BARON:

17         Q.    What does this diagram illustrate, Professor 

18    Wachter?  

19               And if you think it will be helpful, you are 

20    certainly welcome to come on down.

21         A.    Well, this gives a kind of diagramatic 

22    representation of the issues of homogeneity and 

23    heterogeneity as I think about them in my studies of 1990 

24    census adjustment.

25               So each of the rectangles on the left and the 
                                                              2118

 1    right represent an area which is to be adjusted, and each of 

 2    those rectangles is divided both vertically and 

 3    horizontally, it's divided into vertical bars and into 

 4    horizontal rows and each different horizontal row represents 

 5    a different local area, for example, a different census 

 6    district or congressional district in the area.  So the rows 

 7    stand for places and the vertical columns, the bars there, 

 8    stand for different demographic groups separated by age and 

 9    race, for instance.

10               Now, this is in the real setting of census 

11    adjustment, there are a lot of different demographic groups 

12    separated out with post-strata, typically 72 different bars 

13    instead of the six bars on this chart, but you can think of 

14    those different groups as being divisions like divisions 

15    into post-strata.  For the clarity of the diagram, I have 

16    just given six instead of 72.

17               So we have different rows for different places, 

18    different columns for different groups and each of those 

19    little boxes is what we call --  represents in this diagram 

20    a unit which is called a district post-stratum intersection 

21    or if the places were actually blocks we would call it a 

22    plot post-stratum intersection.

23               Now, on the left you have a pattern represented 

24    which is what the homogeneity assumption assumes.  The 

25    gradation from pink to purple in the chart represents 
                                                              2119

 1    gradations of differing net undercount, so the more purple 

 2    on the chart the more net undercount and the more likely 

 3    pink the less undercount, and the homogeneity assumption 

 4    posites that for practical purposes within these 

 5    post-strata, within these groups, there aren't any 

 6    differences that matter from place to place, and that's why 

 7    you see the bars as colored stripes.

 8               The same group has the same color wherever it is 

 9    living and in census adjustment the same group will receive 

10    the same adjustment factor wherever it's living within the 

11    post-stratum.

12               So on the left we have what is being assumed.  On 

13    the right you have what happens if heterogeneity is present 

14    if the homogeneity assumption fails in some practical way. 

15               And there we see that there are lots of different 

16    colors in each of the same bars, so people in the same group 

17    stay with the same race, age, sex, may have different 

18    undercount rates depending on where they live.  So the 

19    effect of that is to produce a kind of patchwork quilt.

20               So this is a stylized representation in this 

21    diagram of the difference between the homogeneity 

22    assumption, says the world is like, sort of like a flag, it 

23    has sort of like simple bars, compared to what would happen 

24    in the presence of heterogeneity, which would be a world 

25    like a patchwork quilt.
                                                              2120

 1         Q.    Is there a more detailed discussion of this chart 

 2    available?

 3         A.    Yes, there is.  I've written a brief report which 

 4    spells out with reference to the chart various different 

 5    kinds of ways in which the heterogeneity assumption might 

 6    fail, and if one goes into detail, which I don't propose to 

 7    go into here, you can recognize different forms of 

 8    heterogeneity illustrated in this chart.

 9         Q.    You have been speaking about heterogeneity and 

10    homogeneity somewhat abstractly.

11               Can you explain what you are referring to with a 

12    concrete example?

13         A.    Well, okay.

14               Let's, let's think about the west south central 

15    division, which includes the states of Texas, Oklahoma, 

16    Arkansas and Louisiana, and what I want you to think of, 

17    consider, well, let's consider two women who are both 25 

18    years old and let's suppose one of them is a Cajun woman who 

19    has just moved to Shrevesport and the other one is now what 

20    we start calling an Angelo woman who, say, lives in Waco, 

21    Texas.  Those two women are going to be in the same 

22    post-stratum so they are going to receive the same 

23    adjustment factor, so they are in two different places on my 

24    diagram, but in the same row and they get treated the same 

25    way by the PES in respect of undercounts.
                                                              2121

 1         Q.    That is, the adjustment formula assigns them the 

 2    same probability of being missed?

 3         A.    That's right.

 4         Q.    Is the homogeneity assumption in the adjustment 

 5    methodology likely to be right?

 6         A.    Well, if you think about examples of this kind, 

 7    you can see that the homogeneity assumption asks a lot.  

 8    Cajun woman in Schrevesport and Angelo women in Waco are not 

 9    very obviously going to be missed or miscounted at the same 

10    rates, but it's a matter of evidence.  

11               It's easily believable that the homogeneity 

12    assumption can fail in important ways or it might be that 

13    for practical purposes it doesn't.  One has to look as best 

14    one can at the actual data such as we have it on variations 

15    within post-strata from place to place.  It's an empirical 

16    question.

17         Q.    Let me ask you this, Professor Wachter:

18               Which is the term in the Bureau's total error 

19    model in the P16 project which gives the Bureau's 

20    measurement of the contributions of errors due to 

21    heterogeneity?

22         A.    There isn't any term in the Bureau's total error 

23    model for these errors.

24         Q.    Well, explain to the court, if you would, what 

25    errors are left out of the so-called total error model.
                                                              2122

 1         A.    Well, the total error model is really a misnomer.  

 2    The total error model is not a model for all the errors, 

 3    it's a model that synthesizes a number of the sampling and 

 4    nonsampling errors that are measured in the P studies.  It 

 5    doesn't include all sources of nonsampling error and, more 

 6    important, it doesn't include these extra errors that I am 

 7    talking about from heterogeneity that come in when you use 

 8    the hemogeneity assumption to go down to local areas and 

 9    back up to states.  So there are errors left out of the 

10    total error model.

11         Q.    Are you on record as having made this point 

12    before in any published work?

13         A.    This is a point that Dr. Terrance Speed and I 

14    made in a commentary on the dress rehearsal version of the 

15    total error model which was published in the Journal of the 

16    American Statistical Association.

17               We begin by pointing out that the total error 

18    model must not be misunderstood as a total error model.

19         Q.    Are errors due to heterogeneity left out in the 

20    Bureau's quantification of error ranges for adjusted counts?

21         A.    Yes.  They are not part of the margins of errors 

22    which the Bureau publishes, for example, the margins which 

23    it published in the 13th of June 1991 press release that all 

24    of us have used.

25         Q.    That is contained as Defendant's Exhibit 51, and 
                                                              2123

 1    I believe this document, which is also Plaintiff's Exhibit 

 2    21, there is a bar chart at the end of that exhibit.

 3               Perhaps you could turn to that.  It's in your 

 4    folder as Defendant's Exhibit 51, the last page of which is 

 5    what I want to ask you about.

 6               (Pause)

 7               Do you have that?

 8         A.    I do.

 9         Q.    Were you here when Dr. Fay testified?

10         A.    Yes, I was.

11         Q.    Dr. Fay testified that the bars on this chart 

12    ought to be wider, did he not?

13         A.    I heard him testify to that effect.

14         Q.    Are you describing the same effect as Dr. Fay?

15         A.    No.  I'm describing another effect.  This is yet 

16    another reason why these bars should be wider.

17               Dr. Fay was discussing variances and he was 

18    stating that the variances or standard deviations for the 

19    smoothed adjustment factors are understated for various or 

20    believed to be understated for various reasons which he 

21    gave.

22               I'm talking about a completely separate effect.  

23    I'm talking about how the bars should possibly also be wider 

24    because of errors that occur because of the hemogeneity 

25    assumption when you carry adjustments down to local areas.  
                                                              2124

 1    So this is extra or the two sources, at least, that may make 

 2    these bars wider.

 3         Q.    Are other numbers that have been presented to the 

 4    court affected by this source of error which you say is left 

 5    out of the Bureau's total error model?

 6         A.    Well, all comparison between adjusted counts and 

 7    unadjusted counts for states or local areas or local 

 8    jurisdictions are potentially affected by these errors.

 9               For example, the results that we have heard 

10    described on loss function are affected by them.  All the 

11    comparison for areas, for geographical areas are potentially 

12    affected by them.

13         Q.    In your opinion, Dr. Wachter, are the effects of 

14    heterogeneity large enough to be important vis-a-vis other 

15    sources of error left out of the total error model?

16         A.    My own research, which I'm prepared to describe 

17    for the court, indicates to me that they are important, that 

18    they are of comparable importance to the sources of error 

19    that are put into the total error model.

20         Q.    What is the importance of the terms left out of 

21    the total error model, including heterogeneity?

22         A.    Well, I guess the chief importance comes in when 

23    we try to construct a hypothetical truth against which we 

24    compare the census on the one hand and the adjustment on the 

25    other hand, and there are many ways of doing that, and one 
                                                              2125

 1    way we heard a lot about in this court is loss functions, 

 2    but this is a point that is not restricted to loss 

 3    functions, it's a general point about comparison, and I 

 4    guess the way I see it is fairly close to something the 

 5    Secretary says in his report about hypothetical truth when 

 6    it's constructed from an idealized dual system estimator, an 

 7    idealized PES.

 8               Nay I explain a little bit what I mean?

 9         Q.    Please do.

10         A.    There are two different settings in which we 

11    might compare adjusted and unadjusted counts to some 

12    hypothetical truth.

13               In one setting we have some external standard for 

14    hypothetical truth which might be subject to error and, 

15    therefore, hypothetical, but it comes from outside, for 

16    example, from demographic analysis if only demographic 

17    analysis were suited.

18               Well, demographic analysis doesn't provide us 

19    that external reading on truth, so in assessing the 

20    relatively accuracy of adjusted and unadjusted counts we 

21    have to get our hypothetical truth in some other way, and 

22    the only way I know how to do it, the only way the Bureau 

23    seems to have known how to do it was to build it up from the 

24    PES itself using the total error model.

25               So the total error model, at its best, gives you 
                                                              2126

 1    information about how far the truth may be different from 

 2    the PES.  It measures, gives us some ideas of the errors, so 

 3    we construct our hypothetical truth by starting with the PES 

 4    and then moving away from it by the amount that the total 

 5    error model says truth may differ from the PES and then we 

 6    get a hypothetical truth.

 7               Now, that's fine as far as it goes, but if there 

 8    are sources of error that are left out of the total error 

 9    model, then we start with the PES and move away from it only 

10    as much as the total error model says, and the source is 

11    left out, then we haven't moved away from the PES enough, so 

12    our hypothetical truth, our idealized DSE may be closer to 

13    the PES than it should be.  

14               If we left out sources of error, we have 

15    introduced an error into our assessment.  We made the 

16    hypothetical truth tend to be closer to the PES and so we 

17    have done something which will bias the results of tests 

18    toward adjustment or tend to bias the tests toward 

19    adjustment.

20               So that's the problem we are dealing about.  

21    Heterogeneity is potentially a big source of error left out 

22    of the total error model and so heterogeneity is a big 

23    potential source of bias when we try to test the census 

24    counts and the PES against our hypothetical truth.

25         Q.    Is heterogeneity the main omitted error?
                                                              2127

 1         A.    Heterogeneity is one of the main omitted errors 

 2    and it's the one I've studied the most, but there are 

 3    sources of nonsampling error or bias that are left out of 

 4    the total error model and there are sources of sampling 

 5    error or variance there are also left out and about which 

 6    Dr. Fay testified, so this isn't the only one, but it's an 

 7    important one.

 8         Q.    Did you hear Dr. Fisher's testimony in this case?

 9         A.    I did.

10         Q.    Were you present in the courtroom when Dr. Fisher 

11    stated the following, and I'm reading from page 971 of the 

12    transcript at lines 13 through 16.

13               "Q.    Is there any respect in which the loss 

14    function analysis is biased or weighted in favor of one or 

15    the other outcome?

16               "A.    No." 

17               Did you hear that statement?

18         A.    I did.

19         Q.    Do you agree or disagree with Dr. Fisher's 

20    statement?

21         A.    Dr. Fisher --  I heard all of Dr. Fisher's 

22    testimony, I heard the testimony leading up to that 

23    question.

24               It seemed to me that Dr. Fisher was discussing 

25    this issue of idealized DSEs and loss function comparison in 
                                                              2128

 1    general somewhat in a theoretical vacuum.  He was discussing 

 2    what would be true if we did have all the sources of error 

 3    that were important included in the total error model, we 

 4    had a sound, a sound estimates for bias and variance, and my 

 5    own views about this question are really shaped by my 

 6    studies of the P projects and my studies of the total error 

 7    model and my studies of the variances for the adjustment 

 8    factors and a number of detailed studies, and those studies 

 9    --  well, they give me a very lively sense of how much isn't 

10    in the total error model.

11               The Bureau did the best job that it could, I 

12    think, under the time constraints that it had, but there is 

13    a lot that simply isn't in the total error model and those 

14    omitted sources of variability or error are going to bias 

15    things like loss function comparison toward favoring the 

16    PES.

17               I think in a realistic setting Dr. Fisher was 

18    wrong.

19         Q.    Could you turn to Plaintiff's Exhibit 9 of the 

20    Secretary's decision.  I have one question related to pages 

21    2-27 and 2-28.

22               My question to you, Professor Wachter, is, does 

23    the Secretary's discussion of the idealized DSE sound 

24    reasonable to you?

25         A.    Let me make clear that this discussion is really 
                                                              2129

 1    not confined to loss functions, it applies to all the kinds 

 2    of comparison we might make for geographical levels, and on 

 3    this key point about the idealized DSE or about the 

 4    hypothetical truth, this passage seems to me eminently 

 5    sensible.  It seems to me more sensible and realistic than 

 6    what Dr. Fisher said.  

 7               MR. BARON:  At this time, your Honor, I move the 

 8    admission of Defendant's Exhibit 41 into evidence and 

 9    Defendant's Exhibit 41 A, which is the chart I would offer 

10    in evidence only for illustrative purposes.  

11               MR. GOLDIN:  Your Honor, I would renew my 

12    objection on the grounds that on the face of it, having had 

13    an opportunity now to compare the photocopy of the chart 

14    appended or what purports to be a photocopy of the chart 

15    appended to Defendant's Exhibit 41 and the colored version 

16    of this chart or the colored version of some formerly  

17    similar chart being offered as 41 A, I simply do not believe 

18    from a visual inspection that they can be the same chart and 

19    I, therefore, think that it is unnecessarily confusing and 

20    inappropriate to have the representation that they are the 

21    same.  

22               THE COURT:  The objection is overruled.

23               Defendant's Exhibit 41 and it's attachment and 

24    the chart are admitted, the chart for illustration only.  

25               MR. BARON:  Your Honor, this would be a natural 
                                                              2130

 1    break point.  

 2               THE COURT:  All right.  Let's break for lunch.  

 3    We will resume at 2:00 o'clock.  

 4               MR. BARON:  Thank you.

 5               (Defendant's Exhibit 41 and 41 A, respectively, 

 6    marked for identification were received in evidence.)

 7               (Luncheon recess) 

 8    

 9    

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2131

 1                         AFTERNOON SESSION
                           _________________

 2                             2:00 p.m.

 3               THE COURT:  Mr. Baron, you may continue.  

 4    KENNETH WACHTER      resumed

 5    DIRECT EXAMINATION (continued)

 6    BY MR. BARON:

 7         Q.    We are heard a lot about 1,392 poststrata.  Could 

 8    you please tell the Court how the 1,392 poststrata are, in 

 9    fact, divided up?

10         A.    The 1392 poststrata are a division of the 
                                                           

11    noninstitutional resident population, partly along 

12    geographic and partly along demographic lines.

13         Q.    What is the geographic element of the 1392 

14    poststrata?

15         A.    There are two tiers to the geographic division.  

16    There as division according to type of place, like central 

17    city, metropolitan area outside of central city, or 

18    nonmetropolitan area.  There is also a division by area into 

19    the nine Census Bureau divisions.  The country is divided up 

20    into nine pieces.

21         Q.    Professor Wachter, I am going to show you a chart 

22    which has been marked as Defendants' Exhibit 60-A.  It is a 

23    color version of a chart which appears at Defendants' 

24    Exhibit 60 in the second binder.  

25               Could you please tell the Court what this chart 
                                                              2132

 1    represents?

 2         A.    Yes.  This shows those nine divisions that are 

 3    incorporated into the Bureau's post-stratification scheme.  

 4    The four colors pertain to the four Census Bureau regions.  

 5    Then if you look --

 6         Q.    Excuse me, Professor.  That is green, yellow, 

 7    red, and blue on the chart?

 8         A.    Green, yellow, red and blue:  northeast, south, 

 9    midwest, and west.  Then each of those regions is divided by 

10    the darker solid line into divisions.  So you see the 

11    western division, the blue, are divided into the Pacific 

12    division, Washington, Oregon, California, and actually 

13    Hawaii and Alaska; and the mountain division, which is the 

14    rest of the blue.  Each of those regions is divided.  

15               There is also a separate division not shown here 

16    for American Indians counted in American Indian sampling 

17    strata, special strata.  Pretty much everything I say in my 

18    testimony leaves out the 12 poststrata for American Indians.  

19    So we are talking about the rest of the 1392 poststrata.  

20    They are divided along these division lines.

21         Q.    So there are 1380 non-Indian poststrata, right?

22         A.    That's correct.

23         Q.    Professor Wachter, will you now turn to 

24    Defendants' Exhibit 44.  I will put a chart up which 

25    represents a color version of the chart which is the last 
                                                              2133

 1    page of Defendants' Exhibit 44, which is marked as 

 2    Defendants' Exhibit 44-A.  

 3               MR. BARON:  I should also say to the Court that 

 4    this chart in memorandum form was previously introduced in 

 5    evidence or identified into evidence as Defendants' Exhibit 

 6    201.  This is a color version of the chart that appears at 

 7    the end of Exhibit 44 as well.  

 8         Q.    Could you briefly describe for the Court, first, 

 9    what the memorandum represents that this chart comes from?

10         A.    Yes.  Defendants' Exhibit 44 is a memorandum or a 

11    report which I wrote, with some advice and assistance from 

12    my consultant Christopher Artel.  This is an analysis of an 

13    experiment which was conducted at the Census Bureau.  It is 

14    an experiment in what is called re-post-stratification.  So 

15    the Bureau had poststrata, and then this is an experiment 

16    with what happens if you try different poststrata.  So it is 

17    another post-stratification, a re-post-stratification.  

18               The Bureau's experiment was conducted by Henry 

19    Woltman and is described in a Bureau memorandum from Henry 

20    Woltman to John Thompson.  I have taken the information in 

21    that memorandum and analyzed its implications.

22         Q.    Could you tell the Court what the differences are 

23    in terms of the breakdown of the country into states that is 

24    represented by this chart?

25         A.    Woltman's experiment is an interesting 
                                                              2134

 1    experiment.  He keeps, as far as he can, all the demographic 

 2    divisions of poststrata the same.  He keeps the place type 

 3    divisions the same.  He just changes this area grouping.  So 

 4    he defines new poststrata with revised state groups.  

 5    Everything is the same except that the groupings on the left 

 6    are replaced by the groupings on the right.  

 7               In choosing these groupings, as Woltman describes 

 8    it in his memo, he tried to group states with an eye to 

 9    enhancing homogeneity.  He tried to group states together 

10    which in general terms might be expected to have people with 

11    more similar undercount rates.  The Census Bureau divisions 

12    on the left are just administrative historical divisions.  

13    So Woltman's attempt is an attempt to get as much 

14    homogeneity as he can.  

15               He didn't do this in a formal way.  He examined 

16    values of five variables that he thought might be indices of 

17    undercount, and then he tried to preserve contiguous groups 

18    and to keep as close to the Bureau's original division as 

19    possible but to separate out states that seemed different 

20    and to group together states that seemed the same.  

21               For example, you see the Bureau's New England 

22    division in Woltman's experiment is split.  You have 

23    Maine/New Hampshire/Vermont split off from Massachusetts/ 

24    Connecticut/Rhode Island.  That seems seems to me an 

25    intuitive thing to do with the differences between the 
                                                              2135

 1    high-income urban areas in southern New England and the more 

 2    traditional lower-income rural areas of northern New 

 3    England.  Those groupings are more likely to lead to better 

 4    homogeneity than if you lump the two together.  That was 

 5    what Woltman was trying to do.

 6         Q.    You may have stated this but I may have missed 

 7    it.  Could you give the Court an example of one of the 

 8    variables that went into the re-post-stratification the way 

 9    Mr. Woltman did it?

10         A.    Woltman inspected values of variables.  One of 

11    the variables was the proportion of housing units that are 

12    rental units rented by a minority householder.  So he tried 

13    to group together states which had similar values of that 

14    variable and four other similar variables.

15         Q.    You have heard Professor Wolter and Professor 

16    Tukey make reference to alternative post-stratifications in 

17    this trial, correct?

18         A.    Yes, I heard their testimony.

19         Q.    What is your view as to whether the Woltman re- 

20    post-stratification represents a reasonable alternative to 

21    the model the Bureau used?

22         A.    I do think it is a reasonable alternative.  I 

23    think there are some clear cases where Woltman's strata are 

24    likely to achieve better homogeneity than the Bureau's 

25    administrative divisions, and I don't see any prominent 
                                                              2136

 1    instances where they are likely to create groups that are 

 2    less homogenius.  So I think that Woltman's strata are at 

 3    least as likely and probably more likely to be homogenius 

 4    than the Bureau's production strata.

 5         Q.    Does Defendants' Exhibit 44 represent your 

 6    independent analysis of the results of the Woltman re-post- 

 7    stratification?

 8         A.    It does.

 9         Q.    Did you write this report?

10         A.    Yes, I did.

11         Q.    How did you analyze the changes from the Bureau's 

12    stratification to Woltman's alternative stratification in 

13    this report?

14         A.    I actually analyzed them the other way around.  I 

15    took the Woltman strata as the baseline.  I said to myself, 

16    this is a stratification which, among the two we are 

17    considering, is the more likely to be homogenius.  So let's 

18    start with Woltman's stratification and then see what the 

19    effect of adopting the Bureau's stratification would be, 

20    what changes does it produce from the Woltman 

21    stratification.  

22               What Woltman looks at are the adjusted counts of 

23    states.  So he is not doing a local level analysis, he is 

24    doing a state level analysis.  That is reasonable because 

25    what he is changing are the state groupings.
                                                              2137

 1         Q.    What are the outcomes of your analysis?

 2         A.    A number of outcomes are shown in my report.  

 3    Perhaps I could refer us all to them.

 4         Q.    Is there a table which would be helpful?

 5         A.    How about Table 1.

 6         Q.    I happen to have it.

 7         A.    On page 4.

 8         Q.    What does this table represent?  

 9               MR. BARON:  This is a blow-up, your Honor.  It is 

10    marked as 44-A, which represents Table 1 in Exhibit 44.

11         A.    Table 1 is actually on pages 4 and 5.  It goes 

12    over to page 5.  

13               These show results from the Woltman experiment, 

14    and they show the ten largest decreases in adjusted counts 

15    and the ten largest increases in adjusted counts that come 

16    from replacing the Woltman strata, the baseline strata, with 

17    the Bureau's strata.  So the column headed "Change" there 

18    gives the change produced by changing from Woltman to the 

19    Bureau poststrata, and the column headed "Resulting 

20    undercount" gives the, rounded to thousands, undercount 

21    produced by the Bureau's strata.  

22               I want to point out that Woltman's experiment 

23    deals directly with the PES data.  It does not involve the 

24    step of smoothing the adjustment factors.  So what we are 

25    comparing here is Woltman's adjusted counts, which are 
                                                              2138

 1    unsmoothed, against the Bureau's unsmoothed adjusted counts, 

 2    what were called the preliminary PES counts or the raw PES 

 3    counts, because we have to compare like to like.  So this is 

 4    the rounded version of the raw adjusted undercounts on the 

 5    right.  

 6               What you see for Texas, the change in state 

 7    groupings from Woltman's to the Bureau's takes away 247,000 

 8    of the people in the adjusted counts, leaving 435,000 people 

 9    as an undercount in the raw adjustment for Texas.  That is 

10    the way to read the table.  Similarly, down below.  For 

11    Florida, the change adds 269,000 people to the adjustment, 

12    giving Florida, with the Bureau's raw adjustment, an 

13    undercount of nearly half a million people.

14         Q.    You have given two examples from this chart.  In 

15    your opinion, Professor Wachter, do you consider these 

16    changes to be large or small?

17         A.    Well, look at them.  Some of these changes 

18    involve a couple of hundred thousand people.  A lot of the 

19    changes on this chart, which are the ten biggest decreases 

20    and increases, are larger than the resulting undercounts you 

21    have.  

22               So I think for Kentucky a change of 42,000 people 

23    in the adjusted count, which leaves only 12,000 people for 

24    an undercount, that is a big change relative to Kentucky.  

25    For Texas, 247,000 is a big change when you are dealing with 
                                                              2139

 1    an undercount of only 435,000 when you are left.  I would 

 2    call these big changes.  

 3               That is especially interesting because some of 

 4    these big changes are happening in places where there are 

 5    big undercounts.  Sometimes it is claimed that the errors in 

 6    adjustment are all concentrated among states where there 

 7    isn't much adjustment, where it really doesn't matter.  Some 

 8    of thos errors may be concentrated in the middle in that way 

 9    and it may or may not matter to the places that are 

10    affected.  But these are cases where the changes are to 

11    considerable extent concentrated where there are big 

12    undercounts.

13         Q.    Are there other findings you make in your report, 

14    this report that we are referring to?

15         A.    I look not only at overall numbers, I look at 

16    percentage changes.  Then I look at what is probably most 

17    important, the changes that are produced in state shares.  

18    There are tables in my report that spell those changes out, 

19    but I think the best way to summarize them is to say that in 

20    a very conventional way of measuring an average amount of 

21    change, that is, measuring the average of the absolute value 

22    of the change, whether it is up or down, the changes that 

23    are produced by the change in strata are, on average, about 

24    50 percent or more as large as the whole changes in shares 

25    produced by the adjustment.  So we are talking about an 
                                                              2140

 1    effect in changing one part of the Bureau's procedure that 

 2    is creating changes that are almost half as big as the whole 

 3    effect that adjustment is about.  So to my mind this is a 

 4    fairly striking set of changes.

 5         Q.    Did you investigate the statistical significance 

 6    of these changes?

 7         A.    I did.  The large amounts of variability in the 

 8    PES which affect all of the adjusted numbers, sampling 

 9    errors that we have heard a lot of discussion about, those 

10    large sampling errors also make it difficult to detect 

11    statistically significant changes.  But my studies show that 

12    there is a statistically significant excess of large changes 

13    from this re-post-stratification.  By my studies, what we 

14    are seeing is not all sampling error.

15         Q.    In your opinion, Professor Wachter, did these 

16    findings have an impact on your assessment of how adjustment 

17    fares under the Secretary's guidelines?

18         A.    One of the guidelines, I believe guideline 3 -- 

19               MR. GOLDIN:  I object, your Honor, on the grounds 

20    that this study postdates or is of even date with the 

21    Secretary's decision.  I understand your Honor's previous 

22    ruling that Dr. Wachter's studies may come in, but in this 

23    case the study is being applied retroactively as evidence 

24    for a decision that had already been made at the time that 

25    the study at the Bureau was done.  
                                                              2141

 1               THE COURT:  Mr. Baron?  

 2               MR. BARON:  Your Honor, this is evidence which 

 3    goes to the issue of robustness and is appropriate for Dr. 

 4    Wachter to comment on whether his assessment is strengthened 

 5    or not by this evidence.  

 6               THE COURT:  Is there any evidence that the 

 7    Secretary was aware of all these things when he made his 

 8    decision, or will there be?  

 9               MR. BARON:  There is certainly evidence that this 

10    kind of re-post-stratification was going on at the Census 

11    Bureau.  Professor Wachter's report obviously postdates the 

12    decision itself.  However, it is appropriate to look at 

13    re-post-stratifications in the context to see whether or not 

14    the adjustment decision is reasonable.  This is one such 

15    piece of evidence.  

16               MR. GOLDIN:  Your Honor, the report in question 

17    by Emmett Woltman is dated July 15th.  It could not possibly 

18    have been reviewed in the process of generating a decision 

19    by the Secretary that was announced that same day.  There 

20    has been no evidence, no testimony, no representation made 

21    by anyone in this case that it was in fact reviewed in the 

22    course of preparing the Secretary's decision.  

23               As a consequence, your Honor, I believe that it 

24    does violence to the notion of prespecification to introduce 

25    at this point studies that were done subsequently and adduce 
                                                              2142

 1    them as evidence for the application of particular 

 2    guidelines by the Department of Commerce before these 

 3    studies were done.  

 4               THE COURT:  Would you read back the very last 

 5    question that triggered the objection. 

 6               (Question read) 

 7               THE COURT:  The objection is overruled.  I will 

 8    take it.

 9         Q.    You may answer.

10         A.    As the question goes to my own assessment -- 

11               THE COURT:  And not the Secretary's.

12         A.    -- and not the Secretary's, guideline 3, as my 

13    memory serves me, did provide that the adjustment needed to 

14    be robust to reasonable alternatives and, post hoc, this 

15    seems to me to be a reasonable alternative to which the 

16    adjustment is not robust.  So from my own view today, it 

17    seems to me to be evidence that an adjustment would not and 

18    is likely not to satisfy guideline 3.  

19               MR. BARON:  At this time, your Honor, I move 

20    Defendants' Exhibit 44 into evidence and the charts that I 

21    hope are properly labeled as 44-A and 44-B -- I will make 

22    sure that they are labeled properly -- into evidence for 

23    illustrative purposes.  

24               MR. GOLDIN:  Subject to the objections which we 

25    made previously and were overruled, we have no objection.  
                                                              2143

 1               THE COURT:  I will take them on the rationale 

 2    that I explained earlier.  44 is in plus the charts A and B 

 3    for illustration. 

 4               (Defendants' Exhibits 44, 44-A, and 44-B for 

 5    identification were received in evidence)

 6         Q.    Professor Wachter, I would like to go on now from 

 7    states to substate areas.  In your opinion, is heterogeneity 

 8    an important potential source of error in adjusted counts at 

 9    substate levels?

10         A.    Yes, it is.

11         Q.    Did any of the P projects examine heterogeneity 

12    at the substate levels?

13         A.    Yes.  Both P-12 and P-15 bear on heterogeneity at 

14    the substate level, and P-12 was the Bureau's principal P 

15    project, principal analysis project devoted to the analysis 

16    of possible heterogeneity and homogeneity.

17         Q.    Do you have any knowledge whether the Census 

18    Bureau has pursued substate analyses along the lines of the 

19    P-12 project since July 1991?

20         A.    I need to make something clear, I think.  The 

21    original plans for the P projects and for P-12, as they were 

22    presented to the Special Advisory Panel and discussed by the 

23    panel, provided for extensive substate analyses.  But 

24    practically all those substate analyses were not actually 

25    carried out in the Bureau's P-12 project.  
                                                              2144

 1               The P-12 report concentrates almost entirely in 

 2    heterogeneity and homogeneity at the state-to-state level, 

 3    not at the substate level.  To my knowledge, the Bureau has 

 4    not pursued analyses along the original lines of the P-12 

 5    project for substate and local areas since the 15th of July.  

 6    If they have, I have no knowledge of it.  

 7               MR. GOLDIN:  Your Honor, I would object and move 

 8    to strike.  I understand that an expert may rely upon 

 9    material that would otherwise be hearsay, but for this 

10    expert to testify about what the Bureau has not done, I 

11    believe, gets into hearsay that is not appropriate for this 

12    or any other expert to testify to.  

13               MR. BARON:  I don't intend to pursue this line 

14    beyond that question, your Honor.  

15               MR. GOLDIN:  I would still make the motion to 

16    strike the last response.  

17               THE COURT:  The motion is denied.

18         Q.    Was the P-12 project, Professor Wachter, based on 

19    PES data?

20         A.    Only to a very small degree.  The main parts of 

21    the P-12 project were based on census data, a special sample 

22    of census data that was drawn for the purpose of the P-12 

23    project, called the P-12 sample, and included on a computer 

24    tape called the P-12 tape.  

25               The reason is fairly simple.  Although the PES is 
                                                              2145

 1    a very big survey at the national level, it is a very small 

 2    survey at local levels.  You might call it mile wide and 

 3    inch deep.  When you get down to local levels, they are very 

 4    small sample size.  That makes it very difficult -- not 

 5    impossible, but difficult -- to study heterogeneity with PES 

 6    data.  

 7               The census includes data for every block in the 

 8    nation, so it is possible to draw a sample of census data 

 9    which is as big as you like.  The P-12 project drew a sample 

10    of 125,000 block clusters that includes a bit over 200,000 

11    census blocks.  So that is about 25 times the size of the 

12    PES sample.  

13               The problem with doing that, though, is that the 

14    census doesn't measure undercount.  The census measures 

15    people.  It is the PES which tries to measure undercount.  

16    So our sample of census blocks, that is to say the P-12 

17    sample of census blocks, doesn't include direct information 

18    on undercount.  

19               What it does include is information on a number 

20    of variables that are believed to be associated, at least to 

21    some degree, with undercount, variables called proxy 

22    variables.  So the strategy in P-12 was to use a big sample 

23    but to study indirect measures to see what light could be 

24    shed on heterogeneity from those indirect measures.

25         Q.    Give the Court an example of what a proxy 
                                                              2146

 1    variable is.

 2         A.    One proxy used in P-12 is the allocation rate.  

 3    An allocation is a census record in which one or more of six 

 4    key characteristics has been imputed, key characteristics 

 5    like race, age, or sex.  If data on those characteristics 

 6    are missing, then the Census Bureau imputes the 

 7    characteristic, and that is an allocation.  If there is one 

 8    such allocation in the record, that record is called an 

 9    allocation.  

10               That is an example of missing data from the 

11    census, and at least in a very general way you might think 

12    that the study of heterogeneity in missing data rates could 

13    be informative for the study of heterogeneity in undercount 

14    rates if you have no other good alternative.

15         Q.    Have you analyzed these proxy variables based on 

16    P-12 data?

17         A.    I have analyzed four main proxy variables based 

18    on P-12 data.

19         Q.    Have you reduced your analysis to a written 

20    report which presents your results?

21         A.    I have.  

22               MR. GOLDIN:  Your Honor, let me ask for one 

23    clarification from counsel.  This concerns the data 

24    underlying Dr. Wachter's report.  

25               THE COURT:  The date?  
                                                              2147

 1               MR. GOLDIN:  The data underlying Dr. Wachter's 

 2    report.  I have not seen the file of P-12 data.  I got a 

 3    representation from Mr. Baron at the lunch break that it had 

 4    been handed over to someone on our side.  I would like to 

 5    have that representation restated.  

 6               In any event, I would object to this testimony in 

 7    light of the fact that even if we have, in fact, gotten the 

 8    data, and I would would like them to confirm that we have, 

 9    we have received it so belatedly, that is, in the midst of 

10    Dr. Wachter's examination, that it is not possible for us to 

11    make any use of it in preparing for his cross-examination.  

12    I believe that the obligation to provide us with the data 

13    underlying the testimony can only be fulfilled if we have 

14    the data in sufficient time and in sufficient form to make 

15    some practical use of it.  

16               On that basis, as I say, we will ask for a 

17    restatement of the representation and will object to the 

18    fact that this testimony is being elicited when we received 

19    disks only today, about 90 minutes ago.  

20               THE COURT:  Mr. Baron, what do they know and when 

21    did they know it?  

22               MR. BARON:  Your Honor, this underlying data that 

23    Mr. Goldin is referring to is all Bureau data and plaintiffs 

24    at any time in this lawsuit could have asked for it.  

25    Second, if Mr. Goldin would like to talk to Mr. Bokar, who 
                                                              2148

 1    is sitting on the far right, Mr. Bokar, I understand, 

 2    received this data -- 

 3               THE COURT:  Is Mr. Bokar the mysterious 

 4    gentleman?  Nice to see who you are. 

 5               MR. BARON:  Perhaps counsel on the other side 

 6    could confer with each other for that.  

 7               Third, Mr. Goldin's objection is clearly 

 8    premature.  We are in the middle of Professor Wachter's 

 9    direct testimony under Rule 705 of the rules of evidence.  

10    The second sentence says, "The expert may, in any event, be 

11    required to disclose the underlying facts or data on 

12    cross-examination."   

13               We certainly have no objection to proffering this 

14    underlying data.  However, I should make mention that we are 

15    centering our questions most directly on what will be 

16    offered as Defendants' Exhibit 42, which is a document 

17    representting Professor Wachter's expert report.  That 

18    report has been available along with all the other reports 

19    of Professor Wachter from May 1st.  For all those reasons -- 

20               THE COURT:  I'm not sure I understand what is 

21    going on here.  If I correctly sense what Mr. Goldin is 

22    talking about, his objection is not an evidentiary one but 

23    one arising out of the rules of pretrial discovery.  He is 

24    claiming you didn't give him this on time and it is unfair 

25    to expect him to handle it now.  
                                                              2149

 1               MR. BARON:  Your Honor, the rules, as I 

 2    understood them to be, were set in this court's pretrial 

 3    orders which set forth that experts would be deposed and 

 4    following the deposition schedule there would be a 

 5    crossexchange of exhibits, including expert reports on 

 6    May 1.

 7               The discussion that we are having here, the 

 8    testimony is directly to Defendant's Exhibit 42, which is 

 9    Professor Wachter's report.  

10               What Mr. Goldin is objecting to is that the 

11    underlying data which form the basis of portions of this 

12    report were not proffered some time long ago.  But as I said 

13    at the outset --  

14               THE COURT:  The question is whether it is covered 
                                                                   

15    by the retrial order.  

16               MR. BARON:  Yes, that is one point, your Honor.

17               And the second point is, as I began in response, 

18    this is all Census Bureau data which plaintiffs could have 

19    asked for.  If they don't have it, they have no one to look 

20    to except to themselves.  

21               MR. GOLDIN:  Your Honor, if I may respond to 

22    that, we have been asking for these data since July 31, 

23    1991.  The record of Dr. Wachter's deposition held March 30, 

24    March 31 and April 1, 1992 is replete with extensive 

25    discussion of these data and extensive requests that we be 
                                                              2150

 1    permitted to inspect these data.

 2               We are now being told that merely by virtue of 

 3    having a summary report from May 1, 1992 we should have been 

 4    put in the position in which we could, for example, inspect 

 5    the calculations that Professor Wachter performed with the 

 6    data, determine whether he added things up correctly, 

 7    whether he performed computations properly and so on.

 8               Obtaining the data, in fact, in full on 

 9    electronic disk whatever it is now, a hundred minutes ago, 

10    certainly does not put us in that position, I submit, for 

11    purposes of cross-examination, and I believe that there is 

12    patent prejudic requiring us to cross-examine Professor 

13    Wachter on the basis of a brief written report in which he 

14    makes extensive representations about what the underlying 

15    data show while precluding us from making use of the 

16    actually files to determine whether his representations are 

17    correct.

18               (Pause) 

19               MR. BARON:  May I say one more thing, your Honor?  

20               THE COURT:  All right.  

21               MR. BARON: 

22               You are going to lose the motion now, talk 

23    yourself right out of it.

24               The objection is overruled.  

25               MR. BARON:  Thank you, your Honor.  
                                                              2151

 1    BY MR. BARON:

 2         Q.    Do you have Defendant's Exhibit 42 in front of 

 3    you?

 4         A.    Yes.

 5         Q.    How do you measure residual heterogeneity in this 

 6    report?

 7         A.    I use a measure of the extent of residual 

 8    heterogeneity, which is the same form of measure that is 

 9    used to measure sampling variability, the same form of 

10    measure that goes into the bars on the 13th of June 

11    memorandum that we have talked about before, and that's a 

12    measure called a standard deviation.

13               The P12 sample size is large enough that I can 

14    compute a standard deviation due to heterogeneity and that 

15    is a measure which reflects how much on average in a 

16    conventional statistical way of measuring that people in 

17    different local areas within the same post-stratum differ 

18    from each other in terms of the value of the variable under 

19    consideration.

20               So I'm using the same form of measure that is 

21    used in the Bureau's margins of error.

22         Q.    What do you compare your measure of heterogeneity 

23    against?

24         A.    Well, my measure is a measure of the residual 

25    heterogeneity left in the post-strata after the Bureau's 
                                                              2152

 1    post-stratification and I compare it against two other 

 2    quantities:

 3               First of all, this is a source variability that 

 4    is left out of the Bureau's margins of error, so I compare 

 5    my measure against the comparable standard deviation for the 

 6    source of variability that is included in the Bureau's 

 7    margins of errors, and that's the standard deviation due to 

 8    sampling.

 9               Now, the P12 sample is a much bigger sample than 

10    the PES, so I need to convert the standard deviation due to 

11    sampling to a level that would be on a comparable basis for 

12    a sample size of the PES, and I call that a PES equivalent 

13    sampling standard deviation.

14               So I'm comparing my measure, which is what is 

15    omitted from the estimates of error, to the comparable 

16    measure of what's included in the Bureau's margins of error.  

17    That's the first comparison.

18               The second comparison I make is to compare my 

19    measure, which is a measure of residual heterogeneity, 

20    what's still there after the post-stratification, with a 

21    measure of the heterogeneity removed by the 

22    post-stratification.

23               The point of the post-stratification was to 

24    control as much heterogeneity as it could, and you can 

25    measure this by looking at the standard deviation of the 
                                                              2153

 1    post-stratumwide rates, and so that's the amount that's been 

 2    controlled and my measure is the amount that hasn't been 

 3    controlled, and that makes a second interesting comparison.

 4         Q.    Can you use your results to say something about 

 5    overall expected errors for adjustments, for example, 

 6    Brooklyn Heights or Flatbush?

 7         A.    No, I can't do that.  I wish I could.  I can't do 

 8    it.  The Bureau, with the P12 tape, is not in a position to 

 9    do it because the relationships between different 

10    post-strata for the same local area are scrambled in the 

11    Bureau's tape.

12               Potentially this could be done with different 

13    Bureau data, but that would probably be a long enterprise.  

14    So I can look at the relative magnitudes of these errors 

15    that go into the estimates for places like Flatbush or 

16    Brooklyn, but I can't look at them in terms of the errors 

17    that come out.

18               Now, you haven't let me say what I find when I 

19    make these comparison and I would be glad to say that if you 

20    give me the chance.

21         Q.    Go right ahead.

22         A.    I find something quite striking.

23               For the kinds of local areas that the P12 tape 

24    pertains to, which are areas with a few hundred sample 

25    people who represent an area with about 7,000 people on 
                                                              2154

 1    average, so we are talking about local units of about 7,000 

 2    people, I find that the standard deviations due to 

 3    heterogeneity, my measure, tend to be larger than the 

 4    standard deviations due to sampling, so what was left out of 

 5    the Bureau's error margins for these measures is actually 

 6    bigger than what was put in.

 7               And I find that they are of roughly comparable 

 8    size, a little smaller but of roughly comparable size, to 

 9    the amount, to the standard deviations for the 

10    post-stratumwide rates, that is to say, to the amount of 

11    heterogeneity that was removed.

12               So I conclude from this that residual 

13    heterogeneity is not negligible.  This isn't some tiny 

14    amount that is left over, these are numbers that are on the 

15    same order of magnitude, give or take, as the other numbers 

16    we are talking about in this context.  

17               This is indirect evidence, this isn't evidence 

18    for undercounts.  I have made efforts to compare some 

19    numbers to PES data, but the sampling errors are too great 

20    to do this for direct measures of undercount.

21         Q.    Does the analysis in this report provide you with 

22    an opinion as to whether the Secretary's concerns about 

23    heterogeneity were sensible or warranted?

24         A.    My analysis really makes those concerns look 

25    foresighted.  It really bears them out.
                                                              2155

 1         Q.    Are there other results in your report in 

 2    Defendant's Exhibit 42?

 3         A.    Yes, there are a lot of other result.  I have 

 4    only spoken to averages that are computed by averaging over 

 5    all the 1,380 non-india post-strata.  I also present tables 

 6    which show the scatter of results for different post-strata 

 7    and I also take up questions of state to state 

 8    heterogeneity.

 9               There are a number of other results, but I have 

10    told you the most important results and the other evidence 

11    supports the conclusion that I just testified to.  

12               MR. BARON:  At this time, your Honor, I move the 

13    admission of Defendant's Exhibit 42 into evidence.  

14               MR. GOLDIN:  In light of your Honor's ruling with 

15    respect to our objections previously and subject to that, we 

16    will have no objection.  

17               THE COURT:  Understood.  It is admitted.

18               (Defendant's Exhibit 42 marked for identification 

19    was severed in evidence.)

20    BY MR. BARON:

21         Q.    Are you familiar with the studies of 

22    heterogeneity that the Bureau did to carry through with the 

23    P12 project?

24         A.    I am familiar with the P12 report.

25         Q.    What do you conclude with respect to the P12 
                                                              2156

 1    study?

 2         A.    There were three main analyses presented in the 

 3    P12 report.  Most of them, most of the parts of those 

 4    analyses were state to state heterogeneity or hemogeneity.

 5               One of the three approaches, in my view, gives 

 6    fairly strong evidence for statistically significant 

 7    heterogeneity from state to state, that is the analysis 

 8    which is calls the first approached in the P12 report and 

 9    the main third approach in the Secretary's report.

10               Now, the question arises whether that  

11    statistically significant heterogeneity is necessarily 

12    practically significant.  

13               I believe the strength of the evidence in the P12 

14    report gives good reason to suppose that it is, but my 

15    further analyses of P12 data have confirmed that it is.  

16    This is not only significant, statistically significant 

17    heterogeneity but practically significant heterogeneity.

18               The two other parts, the two other main parts of 

19    the P12 report have certain difficulties.

20               There is an analysis using PES data, but there is 

21    a mistake, in my view, in the formulas used for the tests of 

22    significance.  This mistake involves something called the 

23    design effect, and in my view the use of this quantity in 

24    the formula is likely to weaken the apparent evidence for 

25    heterogeneity even when heterogeneity is present.
                                                              2157

 1               So by my study of the report, the part that was 

 2    drawn in non-PES data is not very good evidence.

 3               And then there was a third part which has already 

 4    been described, I believe, once to the court in which some 

 5    smoothed factors and predicted factors were compared, and 

 6    that analysis suffers from a defect which is related to the 

 7    defect that I talked about this morning in terms of 

 8    hypothetical truth and idealized dual system estimates and 

 9    that defect is described, to my view, fairly clearly in the 

10    Secretary's report.

11               So that piece of evidence I think doesn't show 

12    anything really, that part of the study.  So the main study 

13    presented in the P12 report constitutes evidence of the 

14    presence of state to state heterogeneity.

15         Q.    Professor Wachter, do you agree with the 

16    Secretary's conclusions concerning the evidence from P12 

17    which are contained at 2-40 of the record, which is 

18    Plaintiff's Exhibit 9?

19         A.    Which sentence do you mean, the sentence, "It is 

20    impossible to conclude from any information the Bureau 

21    presented in P12 that there is not residual heterogeneity 

22    within post-strata"?

23         Q.    You are reading from the paragraph that starts 

24    "Summary"?

25         A.    That's where I am reading the second full 
                                                              2158

 1    paragraph on the page.

 2               Well, I would go, I would go further than the 

 3    Secretary here.

 4               It seems to me that it is possible to conclude 

 5    from the information the Bureau did present in P12 that 

 6    there is a reasonable basis for concern about residual 

 7    heterogeneity within post-strata.

 8         Q.    Would you call the Secretary's conclusions about 

 9    heterogeneity rational?

10         A.    The Secretary, in his report, viewed the evidence 

11    as mixed, but concluded from it that heterogeneity was a 

12    serious source of concern, a serious potential source of 

13    error in the adjusted counts.  That seems to me to be 

14    eminently rational.  It's turning out to be right.

15         Q.    Professor Wachter, is it fair to say that in your 

16    recommendation to the Secretary you took a very different 

17    approach from the undercount steering committee in assessing 

18    the relative accuracy of the adjusted and unadjusted counts?

19         A.    Yes.  Yes.  I took a more exploratory approach.  

20    I tried to present data from a variety of different 

21    directions that bore on different aspects of the decision.   

22    I didn't try to wrap it up into some single summary measures 

23    or a few summary measures, like loss functions.

24         Q.    Was there information that the undercount 

25    steering committee have that you did not have available to 
                                                              2159

 1    you, if you know?

 2         A.    Yes.  The estimates of sampling variances for the 

 3    smoothed adjustment factors arrived too late for me to make 

 4    active use of them and by the time they arrived I had 

 5    already come to realize that the source of those variances 

 6    in the smoothing model was problematic and it wasn't clear 

 7    that the variances could be trusted, so I was not in a 

 8    position to do analyses that made detailed use of those 

 9    variances.

10         Q.    From the outset of your service on the special 

11    advisory panel, what did you understand the Bureau's plans 

12    to be for assessing relative accuracy of adjusted and 

13    unadjusted counts?

14         A.    Well, this was an area which was discussed a 

15    great deal with the special advisory panel, especially 

16    beginning in October of 1989, when the special advisory 

17    panel commented on and suggested revisions to the 

18    guidelines, it also commented on and suggested a number of 

19    revisions to a document which eventually became the 

20    technical operation plans and was eventually published in 

21    the Federal Register in March or April, probably April, but 

22    we discussed it back in October and November.

23         Q.    Excuse me.  That is March or April of 1990 and 

24    November of 1989, correct?

25         A.    Yes, that's correct.
                                                              2160

 1               Now, in that document, in the technical operation 

 2    plans and throughout our earlier discussions of them, 

 3    summary --  ways to summarize the evidence in terms of the 

 4    relative accuracy of adjusted and unadjusted counts were 

 5    really presented I would say in very general terms, more 

 6    like the terms in which I treated them in my report, and 

 7    there was no central role assigned to a small number of loss 

 8    functions in those discussions.  In fact, the emphasis was 

 9    on how to display results from the P16 total error model, 

10    how to organize those results.

11               Now, later in the discussion --  so I was just 

12    talking about October-November-December.

13               During the winter of 1989-90, John Thompson asked 

14    the special advisory panel members a number of times to 

15    think about the question of summary measures, and he floated 

16    the idea of using loss functions in conjunction with other 

17    models and using a variety of loss functions, and these were 

18    discussed, especially in one meeting where John Thompson and 

19    I and Dr. Tukey were the principal speakers.  But they were 

20    discussed in a kind of exploratory context.  

21               So I think it's right to say that loss functions 

22    were under discussion throughout that winter and spring, but 

23    it's not right to say that they were under discussion as a 

24    central or pivotal part of the Bureau's final assessment.

25         Q.    Let me get this straight, Professor Wachter.
                                                              2161

 1               Were you ever informed as a panel member that the 

 2    Bureau had decided to make a loss function analysis central 

 3    to decisions about relative accuracy?

 4         A.    No, I was not.

 5               Had I been given any inkling of that, I would 

 6    have protested with vigor.  I think that would be and was a 

 7    mistake.

 8         Q.    Do you consider a formal reliance on loss 

 9    functions for decision-making compatible with Guideline One?

10         A.    No, I don't think a formal reliance on loss 

11    functions is compatible with my reading of Guideline One.

12               Guideline One establishes a presumption in favor 

13    of the unadjusted census, unless the adjusted census can be 

14    shown to be of superior accuracy.

15               Now, one could built such a presumption into a 

16    loss function, but that is a complicated and difficult 

17    process and that's a process very remote from the kinds of 

18    loss functions that the Bureau was considering and which the 

19    Bureau undercount steering committee did eventually present.

20               So in terms of the kinds of loss functions that 

21    had been talked about years and years before by the 

22    decennial census panel and the kinds of loss functions that 

23    the Bureau eventually used, they don't establish a 

24    presumption in favor of the census and I wouldn't consider 

25    them compatible with Guideline One.
                                                              2162

 1         Q.    Were results from the P16 project presented to 

 2    the special advisory panel?

 3         A.    Yes, they were presented in a meeting at the very 

 4    end of May and the beginning of June as they first became 

 5    available.  They were presented in oral --  in an oral 

 6    presentation by Dr. Thompson and Dr. Mulry and others.

 7         Q.    This is May or June of 1991 now?

 8         A.    Yes, May or June of 1991, two weeks before our 

 9    reports were due.

10         Q.    Was there any indication at that meeting that 

11    loss function results would form a central part of the 

12    presentation of P16 project outcomes to the Secretary?

13         A.    I don't believe there was, no.

14         Q.    Were you surprised at the role given loss 

15    functions?

16         A.    Well, I first found out about the role given loss 

17    functions after the Secretary's decision, when I received 

18    the report and the addendum to the undercount steering 

19    committee, and I must say I was taken aback.  I had no 

20    expectation that loss functions would play as central a role 

21    as they did.

22         Q.    In your opinion, Professor Wachter, did the 

23    Secretary change the rules of the game with respect to loss 

24    functions?

25         A.    Not as I understood those rules from my service 
                                                              2163

 1    on the special advisory panel.

 2         Q.    I want to read a portion of testimony.

 3               Did you hear Professor Tukey testify in this 

 4    action?

 5         A.    I did.  I heard all his testimony.  

 6               THE COURT:  Can you summarize it without reading 

 7    it?  

 8               MR. BARON:  It is four lines, your Honor.  

 9               THE COURT:  I never win with you.

10               Go ahead.  

11               MR. BARON:  I'll try to be good.  

12    BY MR. BARON:

13         Q.    This portion of testimony --  

14               MR. BARON:  Well, I lied, your Honor, I'm taking 

15    after Mr. Sitcov.

16         A.    Shall I try to summarize what I heard?

17         Q.    I want to move on to the P15 project and prior 

18    testimony in this action regarding the P15 project, because 

19    in your earlier remarks you mentioned it as well as P12 that 

20    it bore on the subject of heterogeneity.  It's at 1414 of 

21    the transcript, lines 12 and on.

22               This is Professor Tukey speaking.

23               "The analysis used by the Census Bureau to 

24    provide a standard error for the adjustment factor for a 

25    post-strata makes use of the differences between the 
                                                              2164

 1    observed undercounts in the different blocks involved, as a 

 2    matter of fact, in pieces of block, because the post-strata 

 3    are divided by age and sex and, therefore, only a part of a 

 4    block will contribute to one, but the analysis looks at the 

 5    observed differences which reflect both sampling error and 

 6    heterogeneity, and this is adequately taken count of in the 

 7    calculation of the standard error." 

 8               Did you hear Professor Tukey say that?

 9         A.    I did.  I did.

10         Q.    Now with the court's indulgence, from Professor 

11    Fienberg's testimony, this is at 1555 of the transcript, at 

12    lines 12 through 19, Professor Fienberg stating --  first, 

13    did you hear Professor Fienberg's testimony?

14         A.    Yes.

15         Q.    Here is the passage.

16               "Nowhere in P15 is the word 'homogeneity' or 

17    'heterogeneity' used.  I read it a second time.  I couldn't 

18    find it.  If it is there, it is in some footnote or hiding 

19    in the back of a table.  That is a document that looks at 

20    something else, simply provides no evidence whatsoever about 

21    the homogeneity problem that the Secretary suggests it does 

22    in the opening sentence."

23              In your opinion, Professor Wachter, are these 

24    opinions of Professors Tukey's and Fienberg consistent or 

25    contradictory?
                                                              2165

 1         A.    They are completely contradictory.  They are 

 2    pertaining to heterogeneity is taken into account in the 

 3    sampling errors, and Professor Tukey is saying, as I 

 4    understand him, all the heterogeneity is taken into account, 

 5    and Professor Fienberg is saying the sampling errors have 

 6    nothing to do with heterogeneity.

 7         Q.    Do you yourself have an opinion on the 

 8    relationship between P15 and the issue of heterogeneity?

 9         A.    This is an extraordinarily technical issue to 

10    which I have devoted a lot of attention and I believe it is 

11    clear that heterogeneity is not entirely taken into account 

12    by the sampling variances as Professor Tukey suggested, but 

13    I also believe that there are statistically significant 

14    traces of the influence of heterogeneity in the sampling 

15    variances, so I would say the truth lies midway between the 

16    position Dr. Tukey suggested and the position that Dr. 

17    Fienberg took.

18         Q.    Comparing the views expressed by Dr. Tukey, Dr. 

19    Fienberg and the Secretary, which, in your opinion, do you 

20    believe is the most nearly correct?

21         A.    The Secretary took the view in the middle and 

22    that seems to me the reasonable view.

23         Q.    Professor Wachter, where in the Ericksen et al 

24    joint report which has been entered into evidence as PX 195 

25    do the authors discuss the validity of the synthetic 
                                                              2166

 1    assumption?

 2         A.    Well, as I recall, they discuss it on pages 20 

 3    and 21 of the joint report, and then those sections draw on 

 4    two of the principal appendices, Appendices G and H of the 

 5    report.

 6         Q.    Have you had occasion to independently review the 

 7    material presented at the pages 20 and 21 of the main report 

 8    as well as in Appendices G and H of what has been marked as 

 9    Plaintiff's Exhibit 195?

10         A.    Yes, I have.

11         Q.    Have you reduced your findings and conclusions to 

12    written form?

13         A.    I have.  I have written a report presenting my 

14    findings and analysis.

15         Q.    Can you turn to what is marked as Defendant's 

16    Exhibit 43 and identify it for the court?

17         A.    Yes.  That's the report I wrote on the subject.

18         Q.    Would you please tell the court what are the 

19    major findings and conclusions in Defendant's Exhibit 43 in 

20    a nutshell?

21         A.    Appendices H and G contain some mathematical 

22    results and some empirical analyses based on 1980 census 

23    data, and in a nutshell, my conclusions are that no strong 

24    conclusions for 1990 synthetic adjustment can be drawn from 

25    the material presented here.  The material is interesting 
                                                              2167

 1    material, it forms a background to any studies of 

 2    heterogeneity, but it's not directly relevant to the 1990 

 3    problem.

 4         Q.    Is there anything else you would like to say in 

 5    terms of what light this analysis sheds on how the 

 6    homogeneity assumption faired in the 1990 PES?

 7         A.    I don't think it really sheds much light on that 

 8    and I don't think plaintiffs' witnesses who testified on 

 9    these matters asserted that it did shed much light on the 

10    1990 question.  It's suggestive evidence.  

11               MR. BARON:  At this time, your Honor, I move the 

12    admission of Defendant's Exhibit 43.  

13               MR. GOLDIN:  Let me object, your Honor.

14               I thought I heard the witness saying that this 

15    did not shed much light on the issue of hemogeneity or 

16    heterogeneity in the 1990 PES.  

17               MR. BARON:  Your Honor, Mr. Goldin is objecting 

18    to Appendices G and H, which were presented by Professors 

19    Ericksen et al in this action as their report to the 

20    Secretary.  

21               Professor Wachter has done his own analysis of 

22    Professor Ericksen et al's reports.

23               THE WITNESS:  I wasn't testifying this doesn't 

24    shed light, I was testifying this was an examination of that 

25    material to see whether that material sheds lights.  
                                                              2168

 1               THE COURT:  And it didn't? 

 2               THE WITNESS:  And it didn't.  

 3               MR. GOLDIN:  At the moment Appendices G and H 

 4    aren't being offered, as I understand it, what is on the 

 5    table is Defendant's Exhibit 43, and as I understand the 

 6    testimony that was just given it is the witness' view that 

 7    this exhibit does not shed any light on the question --  

 8               THE COURT:  No.  He just said H and G sheds no 

 9    light and this exhibit demonstrates it.  

10               MR. BARON:  And Appendices G and H are in the 

11    record as Appendix Exhibit 1.  

12               THE COURT:  The objection is overruled.  43 is 

13    admitted.  

14               (Defendant's Exhibit 43 marked for identification 

15    was received in evidence.)

16    BY MR. BARON:

17         Q.    Let me turn to another subject.

18               The Secretary expressly cites your work, 

19    Professor Wachter, in producing computer simulations aimed 

20    at demonstrating local level homogeneity in support of the 

21    Secretary's decision that Guideline Two is not satisfied, is 

22    that correct?

23         A.    That is correct.

24         Q.    It is 2-42 of Plaintiff's Exhibit 9, Professor 

25    Wachter.
                                                              2169

 1               Did you include a discussion of these computer 

 2    simulations in your original report to the Secretary, which 

 3    has been part of Defendant's Exhibit 1 and also has been 

 4    marked as Defendant's Exhibit 39?

 5         A.    Yes, I do.

 6         Q.    Could you briefly describe to the court what the 

 7    state of your research was in doing these computer 

 8    simulations at the time you presented your recommendation 

 9    against adjusting?

10         A.    This represented the outcome of several months of 

11    research with PES data, but the results were still 

12    preliminary and I presented them to the Secretary as 

13    suggestive evidence from preliminary work.

14         Q.    What did your simulations add to the information 

15    otherwise available from the Census Bureau's P12 and P15 

16    projects or to the work of other special advisory panel 

17    members?

18         A.    Well, these computer experiments were based on 

19    selected collections of actual PES data so they involved the 

20    modeling of the synthetic process using evidence of 

21    heterogeneity in the block level and district level PES 

22    itself, and as far as I am aware, they were essentially the 

23    only evidence that was presented to the Secretary which was 

24    based on the PES evidence, so I regarded them as the most 

25    directly relevant evidence that could be mustered as to 
                                                              2170

 1    whether heterogeneity might be a serious problem at local 

 2    levels.

 3         Q.    Did you use all PES data in these computer 

 4    simulations?

 5         A.    No, I only had a very limited time to work with 

 6    the data and I selected a few sets of PES data selected with 

 7    an eye to showing whether there was a problem here.  I had 

 8    no hope of trying to do a study that was representative of 

 9    the United States, I selected a variety of cases which would 

10    allow me to see whether we were dealing with a problem.

11         Q.    Would you describe for the court one subset of 

12    data?

13         A.    The set that underlie the particular experiment 

14    that I presented in my report includes all the blocks in 

15    Type 2 metropolitan areas in the west south central census 

16    division.  That's the division --  well, the chart of the 

17    divisions has left us, but that is Texas, Louisiana, 

18    Oklahoma and Arkansas.

19               So to do this kind of study you need to take the 

20    largest area that is included in single post-strata and the 

21    Bureau's post-strata spanned the whole of each census 

22    division for selected place types, and so those were the 

23    data that I took.  I took all the block level data in the 

24    PES for those place types for that division.

25         Q.    Were you in the courtroom, Professor Wachter, 
                                                              2171

 1    when Professor Fienberg described your computer simulations 

 2    in the following manner, at page 1556 on the transcript, 

 3    line 3:

 4               "It is true Wachter did do a study, he did a 

 5    study which used data for an area in Texas known as 

 6    Arlington." 

 7               Did you hear that?

 8         A.    I heard it.

 9         Q.    Was Professor Fienberg correct in describing the 

10    Arlington study in these terms?

11         A.    No, that is a misrepresentation.  Arlington is 

12    just a code name we used for the site.  These are the whole 

13    set of blocks for the division for these place types in the 

14    division.

15         Q.    What conclusions did you draw in your 

16    recommendation to the Secretary with respect to the 

17    Arlington computer simulation?

18         A.    Well, it's important to bear in mind that I have 

19    the same problem in these experiments that I described in 

20    terms of the loss functions.  I don't have an external 

21    standard of truth, I have to build my hypothetical truth off 

22    of the information in the post-enumeration survey.  I do it 

23    in the simulations in what I consider to be a slightly safer 

24    way than the loss functions do it, but nonetheless I'm 

25    depending on the post-enumeration survey, and there are a 
                                                              2172

 1    number of other features in these simulations that would --  

 2    might be taken to tend on the whole to give results biased 

 3    in favor of adjustment.

 4               In particular, I am only including sampling error 

 5    and errors due to heterogeneity, I'm not making any attempt 

 6    to introduce errors that come from nonsampling errors in the 

 7    PES.  

 8               I don't believe there is any defensible way of 

 9    allocating nonsampling errors down to post-strata, so in 

10    some sense my simulations are looking at what are the 

11    effects of heterogeneity in sampling error in a somewhat 

12    ideal setting where we don't have systematic errors coming 

13    into the adjusted counts.

14               Now, the simulations that I reported to the 

15    Secretary looked at chunks of district offices, so fairly 

16    large local areas, at least represented by relatively small 

17    numbers of PES blocks but fairly large areas, and in the 

18    cases I reported I looked at both the effect of adjustment 

19    on numeric accuracy and of the effect on distributive 

20    accuracy.

21               And if I recall, on average in my experiments, 

22    adjustment improved the districts, the 23 districts and 

23    about, on average, 59 percent of the districts.

24               Now, considering that there were these intrinsic 

25    features of my experiments which I expected to favor 
                                                              2173

 1    adjustment, I took that to be evidence that a substantial 

 2    portion, possibly a majority of local areas, could be made 

 3    worse by adjustment.  

 4               THE COURT:  Let's take a break.  

 5               MR. BARON:  Thank you, your Honor.  

 6               THE COURT:  We will resume in ten minutes.

 7               (Recess) 

 8    

 9    

10               (Continued on the next page.)

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              2174

 1         Q.    Professor Wachter, before the break we were 

 2    talking about your computer simulations on local 

 3    heterogeneity and the conclusions that you drew in your 

 4    recommendation to the Secretary on them.  

 5               Let me ask you, in your view, did the Secretary 
                                                                 

 6    draw the correct inference from the results of your 

 7    preliminary computer simulations as presented in your 

 8    recommendation about heterogeneity?

 9         A.    The inferences the Secretary drew in his report 

10    were in line with the inferences I drew myself.

11         Q.    Have you had occasion to conduct further computer 

12    simulations or experiments on the issue of local 

13    heterogeneity?

14         A.    Yes.  The results that I described were 

15    preliminary, and they were restricted in the form that I 

16    presented to one of the sets of data from the PES that I had 

17    selected, although I had conducted other simulations and 

18    experiments with other sets which led me to expect that the 

19    results I was presenting would not be completely atypical.  

20               Since that time I have refined my programs and I 

21    have performed a variety of experiments with four versions 

22    of the program, both at district and block levels, and with 

23    four different sets of the PES data.  I don't propose really 

24    to go into them unless you insist.  

25               THE COURT:  Nobody insists.  
                                                              2175

 1               MR. BARON:  I promise.

 2         A.    I would say they show a lot of interesting 

 3    structure and diversity, but basically they support the 

 4    results that I presented to the Secretary.

 5         Q.    Have you reduced your further research and 

 6    analysis to writing?

 7         A.    Yes, I have.

 8         Q.    Can you identify Defendants' Exhibit 46?

 9         A.    Yes.  Defendants' Exhibit 46 is a report that I 

10    have written summarizing the results from my ongoing series 

11    of experiments.  An appendix to that report contains more 

12    details, written by Mark Hansen, a statistical consultant 

13    who has collaborated with me in the writing of the programs 

14    and in the running of the experiments.  He and I worked 

15    together on the preparation of the appendix that you see.

16         Q.    Could you summarize for the Court what your 

17    overall conclusions are based on these computer simulations 

18    in Defendants' Exhibit 46?

19         A.    I think they show that heterogeneity and 

20    heterogeneity in conjunction with sampling error make it 

21    likely that there are substantial additional errors in 

22    adjusted counts at local areas in addition to the kinds of 

23    errors measured for evaluation poststrata in the Bureau's 

24    total error model. 

25         Q.    Does your conclusion support or undermine your 
                                                              2176

 1    preliminary research which the Secretary cited in his 

 2    decision?

 3         A.    It supports it.  

 4               MR. BARON:  At this time, your Honor, I wish to 

 5    move Defendants' Exhibit 46 into evidence.  

 6               MR. GOLDIN:  Subject to the objection that we 

 7    have had previously, your Honor, we have no objection to 

 8    this one.  

 9               THE COURT:  Understood.  It is admitted as 

10    Exhibit 46.  

11               (Defendants' Exhibit 46 for identification was 

12    received in evidence)

13         Q.    Professor Wachter, you referred earlier in your 

14    testimony to a category of people called "unreached people."  

15    Do you remember that?

16         A.    Yes.

17         Q.    Could you look at page 20 of your recommendation, 

18    which is Defendants' Exhibit 39.  Let me read three 

19    sentences from this recommendation and ask you a question.  

20               The recommendation says in the third paragraph, 

21    the last paragraph before the Section 1.8, in italics in the 

22    first sentence, "I believe it is better not to attempt any 

23    formal allocation of unreached people to local 

24    jurisdictions.  Table 15 of the P-16 tables which omit such 

25    allowances seems to me much more reliably informative than 
                                                              2177

 1    Table 16, which includes them.  After the uncertainties have 

 2    been quantified, one can then ask what imbalance of 

 3    unreached people would be required to overturn any 

 4    particular conclusion."  

 5               Is this the passage, Professor Wachter, where you 

 6    describe your approach to unreached people?

 7         A.    Yes, it is.

 8         Q.    Can you give a specific example which illustrates 

 9    the point you are making in this passage?

10         A.    Okay, I'll try.  The P-16 project, the Bureau's 

11    total error model, quantifies a number of measured biases or 

12    nonsampling errors in the adjusted counts, in the PES.  We 

13    also have this unmeasured bias that comes from unreached 

14    people.  

15               The sort of thing I have in mind is taking a 

16    specific example in which we might be interested in asking 

17    how many unreached people would there need to be in order to 

18    overturn a conclusion.  So, for example, we could consider 

19    New York State.  

20               Now, New York State actually has an undercount 

21    that is less than the national average.  So New York State 

22    loses share if counts are adjusted.  New York City has an 

23    undercount which is higher than the national average for the 

24    whole country but rather low compared to undercounts for 

25    other large cities like Los Angeles or Houston.  
                                                              2178

 1               The kind of example I have in mind is consider 

 2    New York State.  Take into account the measured biases in 

 3    the PES which would reduce numbers across the country but 

 4    possibly reduce adjusted numbers differentially from 

 5    different places.  Then consider how many unreached people 

 6    one would have to have in New York in excess of numbers of 

 7    other places so that the census shares would turn out to be 

 8    a better estimate for New York State than the adjusted 

 9    shares.  That is the kind of conclusion I'm looking at.  

10               So New York State loses about 266 parts per 

11    million in shares if the census is adjusted.  I ask in my 

12    analysis how much of an excess of unreached people in New 

13    York would you have to be to go halfway up.  If you went 

14    halfway back toward the census, if the truth were halfway 

15    back toward the census, then the adjusted share for New York 

16    would be less accurate than the census share.  That is the 

17    kind of analysis I have in mind.

18         Q.    What, in your view, are the reasons for New York 

19    City's undercount being so low?

20         A.    We don't know.  That is one of the biggest 

21    puzzles in the PES.  There are a couple of reasons that you 

22    might consider.  

23               One is that New York City seems to have had 

24    larger numbers of erroneous enumerations, fabrications, 

25    duplicates, pets, other mistakes in the census that include 
                                                              2179

 1    extra people.  

 2               THE COURT:  Counting dogs?  

 3               THE WITNESS:  Counting dogs, poodles.  

 4               THE COURT:  I thought "pets" was one of your code 

 5    words or something.  PES, PET.  It's like my tax form.  

 6               Go ahead.

 7         A.    So New York had fairly large numbers of erroneous 

 8    enumerations.  It could be that the resulting undercount 

 9    estimate from the PES for New York City is correct because 

10    in spite of large numbers of omissions, it could be that to 

11    some extent the census was self-adjusting.  So when you got 

12    people left out, you also got people duplicated or 

13    fabricated.  So, in a sense, not in the way we would like it 

14    to happen but in a way that could have happened, perhaps 

15    erroneous enumerations essentially compensated and left the 

16    PES share for New York accurate.  That is one possibility.  

17               Another possibility is that New York has 

18    disproportionately many unreached people, people who were 

19    very hard to count, people who were missed by both the 

20    census and the post-enumeration survey, and that in truth 

21    New York City's undercount was not as low as the PES seems 

22    to show, in truth New York State should not be losing share 

23    from an adjustment, and that these unreached people about 

24    whose numbers we have little information and about whose 

25    locations we have no information, perhaps make up the 
                                                              2180

 1    difference.  That is the sort of question that is at the 

 2    heart of the question of the accuracy of adjusted and 

 3    unadjusted counts.

 4         Q.    Why might New York City have more unreached 

 5    people than Los Angeles or Houston?

 6         A.    All arguments of this kind are speculative, and 

 7    we are really talking about what is credible or plausible, 

 8    not what the data can show.  But some reasons that could be 

 9    put on the table aren't hard to find.  

10               Los Angeles has larger numbers of certain kinds 

11    of Hispanics than New York.  It could be that the PES is 

12    more successful at measuring undercounts among groups with 

13    fairly stable or traditional family structures, and it could 

14    be that those are more represented among Hispanics in Los 

15    Angeles.  Could be, could not be.  I am not saying there is 

16    any proof of this, but it is not implausible.  

17               Some of the most hard-hit areas in Los Angeles 

18    tend to be dispersed housing, whereas some of the areas in 

19    New York, to my understanding, that are difficult areas tend 

20    to be high-rise housing.  It might be that the PES has more 

21    success in measuring undercount in such areas or it might 

22    not be.  This is speculation.  

23               But it is not outside the realm of rationality to 

24    think that the PES could be getting the share of New York 

25    wrong and the share of Los Angeles more nearly right.  It 
                                                              2181

 1    could be accurately measuring undercounts in Los Angeles and 

 2    failing to pick up larger proportions of unreached people in 

 3    New York.

 4         Q.    Did you prepare a table, Professor Wachter, which 

 5    illustrates the points you are making?

 6         A.    I prepared a table which helps me think about 

 7    this question.

 8         Q.    If you turn to Defendants' Exhibit 45, the table 

 9    attached to the one page, and we have a blow-up for 

10    illustrative purposes that has been marked as Defendants' 

11    Exhibit 45-A.  

12               MR. BARON:  I should say, your Honor, this is my 

13    last table or chart.  

14               THE COURT:  I don't believe it.  

15               MR. BARON:  It's true.

16         Q.    What does this table show as I say, it is 

17    attached to Defendants' Exhibit 45, your Honor.

18         A.    This is a table I worked with to think about 

19    these questions, and it is very close to tables that are 

20    given in the 13th of June press release that we have all 

21    talked about.  

22               What you see is an ordering of states in the 

23    table and the share changes in parts per million that would 

24    be produced by an adjustment.  California gains 1,959 parts 

25    per million in share if the adjusted counts are used instead 
                                                              2182

 1    of the census.  New York State loses 266 parts per million.  

 2    That is from the minus 266 in that column.  

 3               Then what I calculate in the second column is the 

 4    number of people who would need to be added to the state 

 5    populations all across the board to bring all the shares for 

 6    all the states closer to the census shares than to the 

 7    adjusted shares.  

 8               So there are two effects in here.  I am looking 

 9    at things disregarding sampling error, which is another 

10    source of error.  I am just looking at what happens with two 

11    kinds of error, the measured biases in the PES, which amount 

12    across the nation to between one and two million or more 

13    people, and the unreached people, about whose numbers we 

14    have loose information but who probably range between a few 

15    hundred thousand and several million by different estimates.  

16               So we have a couple of million people negative 

17    that we can take away from states for the measured biases.  

18    We have between some hundred thousand and some million 

19    people to add in.  And these are the numbers of people we 

20    have to give to each state to bring those shares closer to 

21    the census shares than to the adjusted shares.  

22               Then you see the percentages that they form of 

23    the state population in the third column.

24         Q.    Do the proportions of minorities from state to 

25    state have a bearing on this table?
                                                              2183

 1         A.    They have some remote bearing, but it is really 

 2    not the proportions of minorities in states that so much 

 3    matter, at least by conventional wisdom.  It is more the 

 4    presence of urban areas or areas with difficult to count 

 5    people or areas with at-risk populations.  

 6               So Wisconsin, a large part of the state is rural 

 7    and doesn't have many minorities.  But Wisconsin also 

 8    includes Milwaukee, so it is not unreasonable to think that 

 9    there could be substantial numbers of people unreached in 

10    Milwaukee even though the overall proportion of those people 

11    in the state of Wisconsin might not be large.  So it has 

12    some bearing but it is not very direct evidence.  

13               MR. GOLDIN:  Your Honor, a few moments ago the 

14    witness presaged that he was going to be engaging, as he 

15    candidly put it, in speculation.  I believe we are now 

16    entering into if not several steps into that realm, and 

17    would move to strike the last portion of the last answer.  I 

18    would object to further questions that seek to elicit 

19    speculation from this witness about what might conceivably 

20    be possible.  

21               THE COURT:  The motion is granted.  The objection 

22    is sustained.  

23               MR. BARON:  At this time, your Honor, I wish to 

24    move Defendants' Exhibit 45 into evidence.  

25               MR. GOLDIN:  I will object to that, your Honor, 
                                                              2184

 1    on the grounds that -- 

 2               THE COURT:  I am going to sustain that objection.  

 3    This is just a bit too speculative for my blood.  

 4               MR. GOLDIN:  Thank you.

 5         Q.    Professor Wachter, on the 17th of June 1991, you 

 6    recommended to the Secretary that the 1990 census should not 

 7    be adjusted, correct?

 8         A.    That's correct.

 9         Q.    Is that still your view?

10         A.    It is my view.

11         Q.    Have you heard the testimony of plaintiffs' 

12    experts in this case?

13         A.    I have heard the testimony of all of plaintiffs' 

14    experts except Dr. Cain and Dr. Bailar.

15         Q.    Have you heard any testimony in this case that 

16    would alter your conclusion that the 1990 census should not 

17    be adjusted?

18         A.    No, I have not.  

19               MR. BARON:  Your Honor, I have no further 

20    questions of Professor Wachter at this time.  

21               THE COURT:  Thank you, Mr. Baron.  

22               Mr. Goldin?  

23               MR. GOLDIN:  Your Honor, might we have two or 

24    three minutes just to organize the papers here before 

25    starting the cross-examination?  
                                                              2185

 1               THE COURT:  If you will promise me it will save 

 2    ten, I'll do it.  You don't have to keep your promise, just 

 3    say it.  

 4               MR. GOLDIN:  I would be delighted to.  Thank you, 

 5    your Honor.  

 6               (Recess)

 7    CROSS-EXAMINATION 
                        

 8    BY MR. GOLDIN:

 9         Q.    Professor Wachter, in connection with your work 

10    on this case and your present testimony, you are being 

11    compensated, I believe, is that correct?

12         A.    I am being compensated for my time in research.

13         Q.    You are being compensated at the rate of $200 per 

14    hour, is that right?

15         A.    Yes, that is correct.

16         Q.    That compensation is being received by you from 

17    the defendants, is that right?

18         A.    That is correct.

19         Q.    Professor Wachter, I call your attention to a 

20    document which has been placed into evidence as Defendants' 

21    Exhibit 38.  Would you take a look at that and tell me if 

22    that is a current curriculum vitae.

23         A.    Can you explain to me?  You placed two books, two 

24    additional books before me.  Those are -- 

25               MR. BARON:  I placed those books.  The other 
                                                              2186

 1    binder should be up there still.

 2         A.    Excuse me a moment.  Yes.

 3         Q.    Do you have the exhibit, Professor Wachter?

 4         A.    Yes, I do.

 5         Q.    I call your attention to the page 2 of that 

 6    exhibit.  I call your attention to the heading in the middle 

 7    of the page, "Current Professional Appointments."

 8         A.    Yes.

 9         Q.    Can you tell me, are you currently a member of 

10    the Special Advisory Panel on 1990 Census Adjustment, U.S. 

11    Department of Commerce, as indicated on that page?

12         A.    Insofar as I understand the legal status of the 

13    panel under the stipulation and order, I believe that I am.

14         Q.    You are currently a member of that panel.  

15               Dr. Wachter, are you familiar with the 

16    stipulation and order entered in this case July 17, 1989?  

17         A.    I was once familiar with it.  I am not currently 

18    familiar with it.  

19               MR. GOLDIN:  If I may approach the witness, your 

20    Honor?  

21               THE COURT:  Yes.  

22               MR. GOLDIN:  With the Court's indulgence, I have 

23    handed the witness a copy of the stipulation and order 

24    previously entered into evidence as Plaintiffs' 450.  I am 

25    trying to ascertain whether we have another copy to work 
                                                              2187

 1    with if the Court needs one.  I am simply going to read the 

 2    witness one line from this stipulation and order.  

 3               THE COURT:  Go ahead, read it.

 4         Q.    Dr. Wachter, I would call your attention to 

 5    paragraph 7 of the stipulation and order, which appears at 

 6    page 5.  About nine lines down from the top of the page 

 7    there is a sentence that I shall read into the record.  "The 

 8    panel shall be comprised of eight persons, none of whom 

 9    shall be employed by any of the parties hereto, of such 

10    knowledge, judgment, and probity that their judgment and 

11    advice shall be entitled to the utmost respect by 

12    defendants."  

13               Do you see the sentence that I have just read 

14    into the record, Dr. Wachter?

15         A.    I do.

16         Q.    Do you understand that that sentence prevents 

17    you, as a member of the Special Advisory Panel, from 

18    accepting employment by a party to this litigation?  

19               MR. BARON:  Objection.  It assumes facts not in 

20    evidence.  Professor Wachter is not here as a legal expert 

21    on whether the Special Advisory Panel does or does not 

22    continue to exist.  

23               THE COURT:  That is not the question.  Overruled.  

24               MR. BARON:  It is an additional point, your 

25    Honor.  
                                                              2188

 1               THE COURT:  All right.  You switched gears on me 

 2    on the point.  

 3               You may answer that question if you are able to.

 4         A.    Repeat the question, please.

 5         Q.    Is it your understanding that the sentence that I 

 6    just read into the record prohibits any member of the panel 

 7    from accepting employment by any party to this litigation?

 8         A.    I would defer to counsel.  That is not my 

 9    understanding of the sentence, but I am not a legal expert.  

10               THE COURT:  You have answered.

11         Q.    Is it your understanding, Professor Wachter, that 

12    the United States Department of Commerce is a party to this 

13    lawsuit?  

14               THE COURT:  I think he has answered your 

15    question, Mr. Goldin.  

16               MR. GOLDIN:  Your Honor, at this time I believe 

17    that we have established that Dr. Wachter, who understands 

18    that he continues to be a member of the Special Advisory 

19    Panel, which continues in existence, has, in fact, accepted 

20    employment with the defendants in this litigation who are 

21    parties to this stipulation and order.  I believe that 

22    establishes -- 

23               THE WITNESS:  Excuse me.  I am not acting as an 

24    employee.  I am acting as a consultant.  

25               THE COURT:  Let him finish his sentence.  
                                                              2189

 1               MR. GOLDIN:  I believe we have established as 

 2    well that the circumstances of that employment, that 

 3    arrangement, are in contravention of the stipulation and 

 4    order.  The stipulation and order is, I submit, unambiguous.  

 5               THE COURT:  It is hardly unambiguous.  

 6               MR. GOLDIN:  On this point.  I believe, your 

 7    Honor, that it clearly establishes that a member of the 

 8    panel may not accept employment by a party.  I believe it is 

 9    clear that Dr. Wachter's arrangement with the defendants 

10    constitutes his employment by a party.  I believe it is 

11    clear that the panel cannot be terminated without the 

12    consent of the parties.  And I don't believe there is any 

13    dispute that the panel continues in existence at this time.  

14               In light of those facts -- 

15               THE COURT:  I was about to say I don't agree with 

16    much of what you said, your interpretation of the agreement.  

17    I read it as forbidding a party to appoint somebody who 

18    works for it to the panel.  But once they are on the panel, 

19    I don't think they are envisioning that they can never be 

20    employed thereafter.  

21               Be that as it may, even if you were right in your 

22    construction, that wouldn't affect his competence to be a 

23    witness.  It would simply affect his credibility. 

24               MR. GOLDIN:  I think it would go beyond that, 

25    your Honor, and I would submit that the defendants, by 
                                                              2190

 1    virtue of having employed one of the members of the panel 

 2    and having acted in contravention of the stipulation and 

 3    order, should be put at the risk of their own conduct, and I 

 4    would therefore move to strike Professor Wachter's testimony 

 5    and preclude him from testifying in these proceedings.  

 6               THE COURT:  The motion doesn't stand on good 

 7    evidentiary grounds.  If you sequester a witness in an 

 8    ordinary trial and tell him not to sit in the courtroom, and 

 9    he, in violation of that order, sits in the courtroom and 

10    hears all the testimony, may that witness thereafter 

11    testify?  Answer:  In most jurisdictions, yes.  He may be in 

12    contempt, he may go to jail immediately thereafter, do not 

13    pass go, do not collect $200, but he remains a competent 

14    witness and may testify.  

15               So even if he is in violation of this agreement, 

16    something which I am not prepared to say he is, he would 

17    remain a competent witness and his employment would at best 

18    affect his credibility.  I will give it the appropriate 

19    weight.  The motion to disallow his testimony is denied. 

20               MR. BARON:  Your Honor, I just want to make one 

21    point in terms of -- 

22               THE COURT:  You want to tell me how good I was?  

23               MR. BARON:  Yes, your Honor.  

24               Mr. Goldin said there can be no disagreement as 

25    to whether the panel continues to exist.  The parties are 
                                                              2191

 1    very much in disagreement over whether the panel continues 

 2    to exist.  Professor Wachter's views on the subject, I 

 3    submit, are irrelevant.  He is not an expert with a JD 

 4    degree.  

 5               MR. GOLDIN:  I would simply say in response to 

 6    that, your Honor -- and there is no need to go very far with 

 7    this argument -- I would say that I do believe that panel 

 8    members ought to be entitled to some ability to read and 

 9    interpret the stipulation insofar as it was expected to 

10    govern their conduct.  I assume that the defendants would 

11    extend in that respect.  

12               THE COURT:  All right.  You have made your 

13    points.  Let's move on.  

14               MR. GOLDIN:  Thank you, your Honor.  

15    BY MR. GOLDIN:

16         Q.    Professor Wachter, you testified on direct 

17    examination that you had published a recent paper in the 

18    Journal of the American Statistical Association with your 

19    colleague Professor Speed, is that correct?

20         A.    Recent commentary, yes.

21         Q.    Putting aside that article, you have never 

22    published a paper on the subject of census adjustment, have 

23    you?

24         A.    That is not strictly true.  I published a paper 

25    in the Journal of Education Statistics which bore on 
                                                              2192

 1    adjustment in general, and census adjustment was part of 

 2    what I discussed there.

 3         Q.    Is that the only other paper that you have 

 4    published that dealt with the issue of adjustment, however 

 5    tangentially?

 6         A.    Dealt specifically with census adjustment?

 7         Q.    Yes.

 8         A.    Yes, I would say so.

 9         Q.    You have never published a paper in the area of 

10    survey research, have you?

11         A.    That depends on how narrowly or how broadly you 

12    define survey research.

13         Q.    You define -- 

14               MR. BARON:  The witness was speaking, your Honor.  

15               THE COURT:  I don't think he completed his 

16    answer.  Finish your answer.

17         A.    I think that is basically true.

18         Q.    You testified as an expert witness for the 

19    federal government in connection with litigation concerning 

20    the 1980 census, did you not?

21         A.    Yes.

22         Q.    You were a signatory of a statement opposing 

23    adjustment that was submitted in 1988 to Representative 

24    Dimely, is that right?

25         A.    That's right.
                                                              2193

 1         Q.    Your signature on that statement was solicited by 

 2    your colleague David Freedman, is that correct?

 3         A.    That is my memory, yes.

 4         Q.    David Freedman also successfully solicited your 

 5    signature on a subsequent letter that he wrote and sent to 

 6    Kirk Wolter at the Census Bureau opposing adjustment, is 

 7    that right?

 8         A.    I didn't hear all the words in your question.  

 9    Did you say subsequently successfully solicited?

10         Q.    Yes.

11         A.    Or subsequently solicited?

12         Q.    Subsequently successfully solicited.  He asked 

13    for your signature and he got it, is that right?

14         A.    I remember he asked for my signature on at least 

15    one and maybe several occasions.  I didn't remember that I 

16    signed such a letter.  But if you present it to me, we can 

17    determine whether I did or not.

18         Q.    David Freedman recommended you for appointment to 

19    the Special Advisory Panel, is that right?  

20               MR. BARON:  Objection, hearsay.  

21               THE COURT:  Overruled.

22         A.    No, that is not my understanding, although he may 

23    have done so without my knowledge.

24         Q.    Would you be surprised to learn that Dr. Freedman 

25    had testified that he recommended you for appointment to the 
                                                              2194

 1    panel?

 2         A.    No.

 3         Q.    Dr. Freedman recommended you for retention as an 

 4    expert witness for the government in this litigation, is 

 5    that correct?

 6         A.    I don't know that to my own knowledge, but I 

 7    wouldn't be surprised.

 8         Q.    Is that your belief?  I'm sorry?

 9         A.    That wouldn't be my account, but I wouldn't be 

10    surprised if it were true.

11         Q.    Dr. Wachter, would you please turn to page 1 of 

12    your recommendation which has been introduced into evidence 

13    as Defendants' Exhibit 39. 

14               MR. BARON:  Is counsel referring to the cover 

15    letter or the main body of the recommendation?  

16               MR. GOLDIN:  The main body of the recommendation.

17         Q.    Do you have it before you, Dr. Wachter?

18         A.    Yes.  

19               THE COURT:  That is date stamped 0 etc. 147?  

20               MR. GOLDIN:  447, your Honor.  

21               THE COURT:  447.

22         Q.    Under the heading "1.1  General Consideration 

23    Considerations," Dr. Wachter, do you see a paragraph which 

24    reads, "It is important to appreciate the difficulty of the 

25    challenge which adjusting a census for coverage error poses.  
                                                              2195

 1    Adjusting the 1990 census counts is not a matter of changing 

 2    a few national totals, it is a matter of changing the counts 

 3    for 6.8 million blocks based on information from less than 

 4    two-tenths of one percent of them."  

 5               Do you see that, Professor Wachter?

 6         A.    Yes.

 7         Q.    In writing that it was a matter of changing the 

 8    counts for 6.8 million blocks based on information from less 

 9    than two-tenths of one percent of them, you were off by a 

10    little more than two million blocks, were you not?

11         A.    There are only about 4.5 million blocks, occupied 

12    blocks.

13         Q.    Those are the blocks that would be changed, is 

14    that right?

15         A.    That's right.

16         Q.    Let me refer you now to page 28 of your 

17    recommendation.  

18               THE COURT:  Now we are using the numbers in the 

19    upper righthand corner?  

20               MR. GOLDIN:  That's right, your Honor.  This is 

21    the equivalent of Bates stamp 000 etc. 483.  

22               THE COURT:  2.  You said page 27?  

23               MR. GOLDIN:  I have 3.  Page 28.  

24               THE COURT:  I'm sorry.  Page 28 is 483, right.

25         Q.    Do you have that, Professor Wachter?
                                                              2196

 1         A.    Yes.

 2         Q.    I refer you to the last full paragraph on that 

 3    page, which begins, "Results from one CAT experiment are 

 4    shown in Fig. 2.1.  They pertain to adjustments at the level 

 5    of census district offices.  This is a unit of around 

 6    400,000 people represented by around 7,000 P sample people 

 7    in the PES scattered over around 100 sample block clusters."  

 8               Do you see that sentence?

 9         A.    I do.

10         Q.    The CAT experiments are the computer simulations 

11    that you were describing earlier in your direct testimony?

12         A.    That's right.

13         Q.    I call your attention now specifically to the 

14    last sentence that I just read, the one that reads, "This is 

15    a unit of around 400,000 people," etc.  Do you see that 

16    sentence?

17         A.    Yes.

18         Q.    We can agree, can we not, that that sentence is 

19    incorrect?

20         A.    That's right, that sentence is incorrect.

21         Q.    Now would you take a look, Professor Wachter, at 

22    page 12 of your recommendation.  In the last full paragraph 

23    on that page, fourth sentence, you write, "The overall net 

24    adjustment for Detroit, a factor of 1.012, turns out to be 

25    much lower than Chicago, which has a net adjustment factor 
                                                              2197

 1    of 1.051."  Do you see that sentence?

 2         A.    Yes.

 3         Q.    Those adjustment factors are different from the 

 4    ones both raw and smoothed calculated by the Bureau, are 

 5    they not?

 6         A.    They are different.

 7         Q.    In fact, the Bureau's adjustment factors, both in 

 8    the raw and in the smoothed versions, show a higher 

 9    undercount rate for Detroit than for Chicago, is that not 

10    right?

11         A.    That's what the Bureau's figures show.

12         Q.    Nowhere in your recommendation do you indicate 

13    that the Bureau's adjustment factors differ from your own, 

14    do you?

15         A.    It turns out that my figures are on a more 

16    correct basis than the Bureau's figures, and the reason is 

17    an issue, which I discuss in my report.  

18               (Continued on next page) 

19    

20    

21    

22    

23    

24    

25    
                                                              2198

 1         Q.    You don't indicate in your recommendation to the 

 2    Secretary that the adjustment factors you calculated differ 

 3    from those the Bureau calculated, do you?

 4         A.    I do not.

 5         Q.    And, indeed, at your deposition on March 30 and 

 6    31 and April 1 of 1992 was the first time that anyone had 

 7    called to your attention that the factors you calculated 

 8    differed from those produced by the Bureau, is that not 

 9    correct?

10         A.    That's correct.
                                

11         Q.    As of the date of your deposition about two 

12    months ago, you had not discussed your recommendation with 

13    any personnel at the Bureau, is that right?

14         A.    That's essentially correct, yes.

15         Q.    Let me call your attention now to page 7 of your 

16    recommendation.

17               Would you please look at the paragraph that 

18    begins at the bottom of that page, in particular, the first 

19    sentence, which reads, "For black females, nonblack males 

20    and nonblack females, demographic analysis actually finds 

21    fewer than the PES estimates." 

22               Do you see that sentence?

23         A.    Yes.

24         Q.    That is no longer your position that the 

25    demographic analysis finds fewer black females than the PES 
                                                              2199

 1    estimates?

 2         A.    No, this is the point I explained in my direct 

 3    testimony.

 4         Q.    Dr. Wachter, I would call your attention now to 

 5    your paper which the defendants have introduced as an 

 6    exhibit called homogeneity assumptions in 1990 census 

 7    adjustment and introduction.

 8               Do you have it there?

 9         A.    I do.

10         Q.    Number 41 of defendant's exhibits.

11               Professor Wachter, would you take a look at 

12    figure 1 appended to that report.           

13               MR. GOLDIN:  You will see in a moment why I was 

14    so concerned about having a chart where the rectangles were 

15    suddenly changing colors on me.

16         Q.    In this report, Professor Wachter, on page 3, you 

17    tell us that the rectangles in figure 1 are hypothetical 

18    examples based on real data, is that right?

19         A.    That's right.

20         Q.    I won't ask what that means.

21               You also tell us again on page 3 that darker 

22    shading represents heavier net undercounts and lighter 

23    shading represents lighter net undercounts, is that right?

24         A.    Presumably that's so.

25         Q.    Do you have an understanding how to interpret 
                                                              2200

 1    figure 1?

 2         A.    Looks to me as if the figure in the book, now 

 3    that you call my attention to it, may be upside down from 

 4    the figure that was the colored diagram.

 5         Q.    That would explain a great deal.

 6               Can we resolve that by taking a little sticker 

 7    that says Defendant's Exhibit 41 A off the diagram and 

 8    moving it to the diagonally opposite corner?  

 9               MR. GOLDIN:  Can we stipulate?  

10               MR. BARON:  I will stipulate that for the 

11    purposes of the hypothetical --  

12               THE COURT:  Do you have that other diagram?  

13               MR. BARON:  Sure.  

14               THE COURT:  Put it up there so I can see what you 

15    are talking about.

16               (Pause) 

17               MR. BARON:  Your Honor, the bars here may be 

18    reversed in this diagram, but for the purposes of the 

19    hypothetical, I will be happy to get into this on redirect, 

20    for purposes of the hypothetical --  

21               THE COURT:  Don't say that.  Let's resolve it 

22    now.  

23               MR. BARON:  Counsel asked me to stipulate to 

24    something that I was going to stipulate to something.  

25               THE COURT:  Well, do it.  
                                                              2201

 1               MR. BARON:  Okay.  

 2               MR. GOLDIN:  The stipulation is that we are going 

 3    to turn the chart upside down, right?  

 4               MR. BARON:  No.  That the bars are just simply 

 5    reversed from the diagram that is in the memorandum.  

 6               THE COURT:  If it turns upside down it becomes 

 7    correct?  

 8               MR. BARON:  No, it's flipped.  

 9               THE COURT:  It's flipped?  It's not the only 

10    thing in this room that is flipped. 

11               THE WITNESS:  I think if it is upside down it's 

12    correct, but we would have to compare the numbers to be 

13    certain.  

14    BY MR. GOLDIN:

15         Q.    Well, speaking of comparing the numbers, 

16    Professor Wachter, where are the numbers?  They are not 

17    presented in this report, are they?  

18               THE COURT:  Hold on a second.

19               Is the chart in figure 1 correct? 

20               THE WITNESS:  Well, both charts are correct 

21    because nothing depends on whether they are right side up or 

22    upside down for the point I'm making.  The question is 

23    whether the colored diagram agrees with the diagram in 

24    figure 1, and I now believe that it's upside down from the 

25    diagram in figure 1.  
                                                              2202

 1               THE COURT:  My simple suggestion would be to 

 2    strike the chart, forget about it, and I promise you I will 

 3    never think about it again.  

 4               MR. GOLDIN:  My pleasure, your Honor.  

 5               MR. BARON:  That would be fine, your Honor.  

 6               THE COURT:  It is done.  

 7               MR. GOLDIN:  Thank you, your Honor.  

 8               THE COURT:  You may take it down.

 9         A.    You had asked me where the numbers are?

10         Q.    You don't give us in this hypothetical example in 

11    this report the undercount rates except insofar as they are 

12    hinted at by the shading, is that right?

13         A.    Sorry, I thought there was a question pending 

14    before you asked that, which is where the numbers are.

15         Q.    Are the numbers in this report?

16         A.    I'll answer the question where the numbers are.

17               These are numbers that are directly calculated 

18    from the data diskett that we proffered to you last week.  

19    Those are the local data and they contain these essential 

20    numbers.

21         Q.    You are referring to the diskett --  I'm sorry.

22         A.    The numbers themselves are not in the report, but 

23    they are readily derived simply by adding up columns of 

24    numbers in the data diskett that you have.

25         Q.    They were received Friday evening?
                                                              2203

 1         A.    They were received Friday, yes.

 2         Q.    And this is, in any event, as you indicated, a 

 3    hypothetical example, is that correct?

 4         A.    That's correct, yes.

 5         Q.    Isn't it critical to know the numbers of people 

 6    in these cells in order to determine whether there is enough 

 7    heterogeneity to be worth worrying about or, indeed, any 

 8    heterogeneity at all?

 9         A.    Well, the numbers of people in the cells bear on 

10    the evidence for heterogeneity.  I'm not presenting a 

11    quantitative argument in this document, I'm trying --

12         Q.    And you would need to know those numbers in order 

13    to know whether there was heterogeneity, isn't that true?

14         A.    In what connection, for what argument?

15         Q.    In order to be able to determine whether this is, 

16    indeed, an illustration of heterogeneity?

17         A.    No.  It's an illustration of heterogeneity.  You 

18    are asking wouldn't we need to know numbers if we wanted to 

19    draw conclusions about whether that heterogeneity was enough 

20    to matter in practice for adjusted counts, and then we would 

21    need to know numbers.  But this is an illustration of what 

22    heterogeneity is like.

23         Q.    Can you determine there is heterogeneity in 

24    looking at the undercount rates without looking at the 

25    underlying numbers?
                                                              2204

 1         A.    May I have the question back exactly, please? 

 2         Q.    If can --

 3         A.    The question in my mind is whether you are asking 

 4    about heterogeneity is practically important or not or 

 5    whether heterogeneity is present.

 6               You can determine heterogeneity is present 

 7    without necessarily having the numbers in the cells, but I 

 8    agree, the numbers in the cells are useful when you want to 

 9    know whether practically important amounts of residual 

10    heterogeneity are present.  You would want numbers in some 

11    form to draw firmer conclusions.  

12               MR. GOLDIN:  If I might have the court's 

13    indulgence, can I have a brief use of the flip chart for a 

14    graphic illustration, your Honor?  

15               THE COURT:  Of the one that we knocked out?  

16               MR. GOLDIN:  No, of one of these charts for 

17    writing on.  

18               THE COURT:  Yes, sure.

19               You were talking about flipping.

20               (Pause) 

21         Q.    Professor Wachter, in some of your computer 

22    simulations, you were dealing with undercount and 

23    heterogeneity at the block level, isn't that correct?

24         A.    Yes.

25         Q.    Let's consider an example of three blocks, and 
                                                              2205

 1    with your permission, Professor Wachter, let us call our 

 2    demographic groups the blues and greens for a change.  

 3               Is that all right? 

 4               Blues on the left, greens on the right, blocks 1, 

 5    2, 3.

 6               Suppose for the blues we have the following 

 7    undercount rates (indicating)

 8               Suppose for the greens we have the following 

 9    (indicating).  

10               We have given the blues undercount rates of five 

11    percent in blocks 1 and 2, zero percent in block three.

12               Greens have undercount rates of 10 percent in 

13    blocks 1 and 3, zero percent in block 2.

14               Is the hypothetical clear, Professor Wachter?

15         A.    Yes.

16         Q.    Can you tell from the information that I have 

17    provided so far that there exists heterogeneity?

18         A.    Strictly speaking, no.  It's possible that the 

19    zero percent undercount rates are in cells that are empty.

20         Q.    So from this information alone, you can't tell 

21    whether there is heterogeneity or not?  You don't provide 

22    any information about the numbers of people in the 

23    hypothetical example that you construct from the data in 

24    this report?

25         A.    Well, in this case the colors down the bar within 
                                                              2206

 1    groups are not all the same and if the only differences were 

 2    the occurrence of zero cells, you couldn't get a patchwork 

 3    quilt that looked like this, so I think what I said was true 

 4    in a narrow sense, but the point that you are trying to make 

 5    is also certainly true.

 6         Q.    Actually, if we look at the one chart that we are 

 7    left with, figure 1, most of the cells on the right are, in 

 8    fact, white, are they not?

 9         A.    In this shading, yes.

10         Q.    Now let's go to figure 2 in this exhibit, if you 

11    would, on the following page.

12               Here you have given an illustration of synthetic 

13    adjustment creating a problem, is that right?

14         A.    I think so, yes.

15         Q.    Are you familiar with figure 2?

16         A.    I am familiar with it.

17         Q.    And is it designed as a hypothetical illustration 

18    of a situation in which synthetic adjustment would make the 

19    uncorrected census less accurate?

20         A.    Yes, it is.

21         Q.    Again, with the court's permission, I would like 

22    to use the flip chart briefly.  

23               THE COURT:  Sure.

24         Q.    In figure 2, Professor Wachter, you were dealing 

25    with five blocks, isn't that correct?
                                                              2207

 1         A.    Yes.

 2         Q.    And, again, we have two demographic groups and, 

 3    again, with your permission I would like to denominate them 

 4    blues and greens.

 5               If you don't mind I will call the blocks A, B, C, 

 6    D and E, to be creative.

 7               Now I would like to translate this hypothetical 

 8    that you have proposed as an instance where synthetic 

 9    adjustment would make the original count less accurate into 

10    gross census error rates which would include gross omissions 

11    and erroneous enumerations, and for purposes of this 

12    illustration I would define the gross census error rate as 

13    the absolute value of one minus the census count over the 

14    true count.

15               Are you with me?

16         A.    Cell by cell?

17         Q.    Right.

18               Okay?

19         A.    Why don't you write that down at the bottom.

20         Q.    Absolute value of one minus the census count over 

21    the true count, and in your example you have posited a true 

22    count so here, unlike most of life, we know what the truth 

23    is.

24         A.    Okay.

25         Q.    All right? 
                                                              2208

 1               In your illustration, this gross census error 

 2    rate is going to be zero for all these cells, the first 

 3    eight cells in the chart representing the blues and the 

 4    greens in blocks A, B, C and D, is that right?

 5         A.    Right.

 6         Q.    Then when we get down to block E, the gross 

 7    census error rate for blues is going to be 25 percent, for 

 8    greens it's going to be 17 percent.  

 9               Is that right?

10         A.    That's right.

11         Q.    So this is a hypothetical situation, is it not, 

12    in which all of the census error is occurring in block E, is 

13    that right?

14         A.    Right.

15         Q.    And, in fact, of the ten people counted in block 

16    E, there were two mistakes, meaning that 20 percent of the 

17    count in block E was erroneous, right?

18         A.    Essentially so, yes.

19         Q.    Of the entire count of blues in figure 2, there 

20    were 34 blues in truth, there was one error in overcount, so 

21    the gross census error rate for the blues was about three 

22    percent, is that right, roughly, although I'm sure I can 

23    roughly up a calculator if you needed it?

24         A.    Roughly, yes.  

25               MR. BARON:  If Professor Wachter needs to do some 
                                                              2209

 1    calculation, I'm sure he will let the court know.

 2         A.    Roughly three percent.

 3         Q.    Okay.

 4               For the greens, in the meantime, we have 16 

 5    greens in truth, there was one error was an undercount, so 

 6    the gross census error for the greens was 6.25 percent, or 

 7    thereabouts; is that roughly right?

 8         A.    Right.

 9         Q.    So this is a situation in which knowing that 

10    someone is blue or green does very little to allow one to 

11    predict that the person will be affected by gross census 

12    error compared with whether or not you know that they are in 

13    block E, is that right?

14         A.    Yes, this is an illustration of that situation.

15         Q.    And your point is that if that's what the world 

16    looks like, if all that makes a difference is whether or not 

17    you are in a particular block, not what demographic group 

18    you are in, that it wouldn't make much sense to stratify and 

19    to synthetically adjust on the basis of demographic groups, 

20    is that right?

21         A.    Well, it's not all that makes a difference is the 

22    block, but in this case most of what makes the difference is 

23    the block, and that's an example where this kind of 

24    synthetic adjustment has real difficulties.

25         Q.    Dr. Wachter, in your testimony and in the report 
                                                              2210

 1    that you authored and that defendants have submitted as an 

 2    exhibit, you describe what you call CAT simulations, is that 

 3    right?

 4         A.    Yes.

 5         Q.    And in those simulations, which I believe you 

 6    described as particularly strong evidence of heterogeneity, 

 7    census counts are drawn from certain PES blocks and combined 

 8    with gross omissions and erroneous enumeration rates from 

 9    other PES blocks, is that right?

10         A.    Yes, that is essentially right.

11         Q.    The blocks you were using for the study come from 

12    different kinds of places in different parts of the country, 

13    which is what you call in your terminology different sites, 

14    is that right?

15         A.    I think you have misspoken yourself this one 

16    point.

17               This is the source of the hypothetical true 

18    counts that I use in the experiments and the census counts 

19    are then generated from the hypothetical true counts using 

20    the erroneous enumeration and gross omission rates, but it 

21    is the census counts from the other sites are used for the 

22    hypothetical true counts, not for the census counts.

23         Q.    And you are doing that for target blocks in each 

24    of those four different sites, is that right?

25         A.    Sounds right.
                                                              2211

 1         Q.    You have basically been using four sites in the 

 2    CAT simulations, right?

 3         A.    One of those sites is a combined site which 

 4    combines Detroit and Chicago, but I have been working with 

 5    four sites in that sense.

 6         Q.    Let's make clear what we are talking about.

 7               One site is New York City, is that right?

 8         A.    PES blocks in central city New York.

 9         Q.    Okay.

10               Central city New York is New York City, right?

11         A.    Yes.

12         Q.    The second site is a combination of Chicago and 

13    Detroit that you call Detago, right?

14         A.    Yes.

15         Q.    The third cite is central cities and noncentral 

16    cities in Type 2 metropolitan areas in Texas, Oklahoma, 

17    Louisiana and Arkansas which you call Arlington because it 

18    might include Arlington, right?

19         A.    Right.

20         Q.    And the fourth and last site is noncentral cities 

21    in metropolitan areas other than Los Angeles in California, 

22    Hawaii, Alaska, Washington, Oregon and Utah, which is the 

23    site that you call Berkeley because it might include 

24    Berkeley, California.

25               Have I got that right?
                                                              2212

 1         A.    Sounds right.

 2         Q.    One reason you chose New York City as the site 

 3    was because your own childhood was spent in the environs of 

 4    New York City, right?

 5         A.    Right.

 6         Q.    One reason you chose Detago, as you called it, 

 7    was your wife's childhood was spent in the environs of 

 8    Detroit?

 9         A.    Right.

10         Q.    And you chose Arlington because one of your 

11    students in one of your classes had grown up in Arlington, 

12    Texas, right?

13         A.    From among other possibilities I chose that one 

14    partly for that reason.

15         Q.    And I guess your personal connection to Berkeley 

16    is obvious, right?

17         A.    Yes.

18         Q.    Now, the point of the CAT simulations is to 

19    generate hypothetical true counts and then hypothetical 

20    census counts and adjustments, is that right?

21         A.    Right.

22         Q.    And the way that you do that is by taking a 

23    target block --  is that a term we can use here?

24         A.    Okay.  That's not my term, but I think --

25         Q.    What is your term?
                                                              2213

 1         A.    Is it?

 2         Q.    What is your term?

 3         A.    Go on with your sentence.

 4         Q.    I just want to make sure you understand what I am 

 5    saying.

 6         A.    The term sounds okay to me.  Keep going.

 7         Q.    Okay, let's adapt it.

 8               You take a target block, say a block in New York 

 9    City, with a certain number of, let's say, black males 

10    between the ages of zero and nine and for a given simulation 

11    you replace the number of black males between the ages of 

12    zero and nine on that block in New York City with a number 

13    of black males between the ages of zero and nine from a 

14    donor block located in the same bin as the target block.

15         A.    That's right.

16         Q.    Is that right?

17         A.    Right.

18         Q.    And the bin would comprise all blocks in all four 

19    sites that are, let's say for purposes of this example, 

20    between zero and 20 percent minority.  Is that right?

21         A.    That's right.  You are discussing, describing the 

22    simulation program in the version that went into this, my 

23    recommendation to the Secretary.

24         Q.    So the number of black males between the ages of 

25    zero and nine for that block in New York City could wind up 
                                                              2214

 1    being determined for some given simulation by the number of 

 2    black males between the ages of zero and nine on a block in 

 3    Portland, Oregon or Baton Rouge, Louisiana, right?

 4         A.    Yes.

 5         Q.    Or Shrevesport or Waco, to take the examples you 

 6    gave earlier?

 7         A.    Right.

 8         Q.    And at the same time for the same block for the 

 9    same simulation, the number of black females between the 

10    ages of zero and nine might be determined by the number of 

11    black females between the ages of zero and nine on a block 

12    in Chicago; is that right?

13         A.    That's a feature of the program used in my 

14    recommendation to the Secretary.  I have since varied that 

15    features in simulations recorded here.

16         Q.    So for some versions of the simulations, that's 

17    the way it would work?

18         A.    That's true.

19         Q.    And that is true of the version you gave the 

20    Secretary, is that correct?

21         A.    That's correct.

22         Q.    And for the same block for New York City, you 

23    might borrow the number of Hispanic males between the ages 

24    of zero and nine from another block in New York City?

25         A.    These are all for constructing the hypothetical 
                                                              2215

 1    true counts.

 2         Q.    Right.

 3         A.    We haven't gotten to the census or the adjustment 

 4    yet.

 5         Q.    I understand.

 6               And the number of Hispanic females between the 

 7    ages of zero and nine might come from the very same block as 

 8    you are using as the target, right?

 9         A.    That's right.

10         Q.    So we could have a situation where the target 

11    block that started this whole process at five black males 

12    between the ages of zero and nine, five black females 

13    between the ages of zero and nine, five Hispanic males 

14    between the ages of zero and nine and five Hispanic females 

15    between the ages of zero and nine, and when the simulation 

16    is run, the equivalent truth for that block that you are 

17    using is zero black males between the ages of zero and nine, 

18    zero black females between the ages of zero and nine, 20 

19    Hispanic males between the ages zero and nine and five 

20    Hispanic females between the ages of zero and nine; that 

21    could happen, could it not?  

22               MR. BARON:  Can I have that question reread, your 

23    Honor?  

24               MR. GOLDIN:  I can repeat it.

25         Q.    So we could have a situation, Professor Wachter, 
                                                              2216

 1    where the actual PES block that started this whole process 

 2    had five black males between the ages of zero and nine, five 

 3    black females between the ages of zero and nine, five 

 4    Hispanic males between the ages of zero and nine and five 

 5    Hispanic females between the ages of zero and nine and when 

 6    the simulation is run, the simulated truth for that block 

 7    could be in the computer zero black males between the ages 

 8    of zero and nine, zero black females between the ages of 

 9    zero and nine, 20 Hispanic males between the ages of zero 

10    and nine and five Hispanic females between the ages of zero 

11    and nine?  That could happen, right?

12         A.    Well, it's not the simulated truth for that 

13    block, it's the hypothetical truth that we used to begin the 

14    simulation process --

15         Q.    I'm sorry, I misspoke.

16         A.    But in describing the targets, what you said is 

17    correct.  That is something that can come out in describing 

18    what you are calling the targets.

19         Q.    That could be the hypothetical truth for the 

20    simulation?

21         A.    That could be the hypothetical truth with which 

22    we begin the simulation.

23         Q.    So that when you create a hypothetical CAT block, 

24    there is no actual block in the real census or the real PES 

25    or anyplace outside the computer that has the same 
                                                              2217

 1    population with the same gross omission and erroneous 

 2    enumeration rates, is that correct?

 3         A.    No, that's not correct.  You have been describing 

 4    the hypothetical truth, but the erroneous enumeration rates 

 5    and the gross omission rates are taken from the PES data.  

 6    They are estimated from the PES data.  That's for the next 

 7    stage of the simulation.

 8         Q.    Okay.  I am talking now about the hypothetical 

 9    true block to which those rates were applied.

10               There is no --

11         A.    I think what you are saying is actually correct.

12         Q.    --  actually block that corresponds to that --

13         A.    There is no actual block that corresponds to that 

14    hypothetical block, that's correct.

15         Q.    The CAT simulations were designed, were they not, 

16    to offer evidence on the extent of local level heterogeneity 

17    in the PES?

18         A.    Yes.

19         Q.    In the CAT simulations reported in your 

20    recommendation to the Secretary as you described them, 

21    numeric accuracy for districts was found to be improved by 

22    adjustment about twice as often as it was found to be 

23    worsened by adjustment, is that right?

24         A.    More or less so.

25         Q.    You can take a look at Exhibit 39, page 28 for 
                                                              2218

 1    the reference.

 2               (Pause)

 3         A.    That's numeric accuracy.  Yes, that's right.

 4         Q.    And for those same simulations, when it came to 

 5    distributive accuracy, on average, 59 percent of districts 

 6    showed improvement and in 93 out of a hundred trails a 

 7    majority of the districts were improved by adjustment, is 

 8    that right?

 9         A.    That's right.

10         Q.    Under the design of the CAT simulations as you 

11    have described it, even if there were perfect homogeneity, 

12    you would not have 100 percent of districts improved, is 

13    that right?

14         A.    That's right.

15         Q.    You have concluded from the set of CAT 

16    simulations recounted in your recommendation to the 

17    Secretary that the fact that 59 percent of districts showed 

18    improvement from adjustment means that heterogeneity is a 

19    problem, is that right?

20         A.    That's right.

21         Q.    But you cannot say what percentage of districts 

22    would have to show improvement for you to determine that 

23    heterogeneity is not a problem, is that right?

24         A.    I had in mind and have in mind no threshold.

25         Q.    Meaning that there is no specific number that 
                                                              2219

 1    would indicate to you that heterogeneity is not a problem, 

 2    is that fair?

 3         A.    That's fair.

 4         Q.    So when a hypothetical district in the CAT 

 5    simulations is taken further from the hypothetical truth by 

 6    the hypothetical adjustment, that could be because of random 

 7    error or because of heterogeneity or because of both, is 

 8    that right?

 9         A.    That's right.

10         Q.    And that random variability is introduced by the 

11    design of the experiment, is that right?

12         A.    Well, yes.  It reflects sampling variability, the 

13    kind of variability that comes into PES adjustment factors.

14         Q.    And in the simulation that comes from the way in 

15    which you put together the hypothetical truth from 

16    components of other blocks, is that right?

17         A.    No, that's not quite right.  That --

18         Q.    I was doing pretty well.

19         A.    The variability comes, really comes in after we 

20    use the gross omission and erroneous enumeration rates that 

21    are estimated from the PES data and then those are rates and 

22    then we do a random realization of those rates, so we toss a 

23    coin as to who in this process in the computer is actually 

24    omitted or who is actually duplicated and that's where the 

25    main part of the random variability comes in.
                                                              2220

 1         Q.    And you are also randomly selecting components 

 2    from other PES blocks to constitute the hypothetical truth?

 3         A.    I am also doing that, but from a certain point of 

 4    view --  there is a technical consideration.  From a certain 

 5    point of view these results are conditional on that choice, 

 6    but from a certain other point of view they are not.  So you 

 7    are in a rather technical area, but what you are saying is 

 8    basically true.

 9         Q.    Okay.

10               So you have not, in fact, determined how much 

11    improvement would be achieved by adjustment under conditions 

12    of perfect homogeneity, is that right?

13         A.    Not, not for the simulations in my 

14    recommendation, not for these simulations.

15         Q.    So it would be material that you presented to the 

16    Secretary in your recommendation did not include a 

17    determination of how much improvement would be achieved by 

18    adjustment under conditions of perfect homogeneity, is that 

19    right?

20         A.    That's right.

21         Q.    There was no baseline given in that 

22    recommendation against which one could judge whether 

23    adjustment was doing close to what would be expected with 

24    perfect homogeneity, is that right?

25         A.    That's right, yes.
                                                              2221

 1         Q.    Okay.

 2               Now, in the CAT experiments that you have 

 3    described with your colleague, Mr. Hansen, in the exhibit 

 4    that has been received into evidence as Defendant's Exhibit 

 5    46, do you deal further with the question of what happens 

 6    with homogeneity?

 7         A.    I do.  We do.

 8         Q.    Is there a version of the simulations that you 

 9    called Fit 1?

10         A.    There is.

11         Q.    And is Fit 1 supposed to describe what happens 

12    with perfect homogeneity?

13         A.    Yes.  Fit 1 includes sampling error, but it 

14    doesn't include systematic heterogeneity.

15         Q.    Would you take a look at page, well, it's 

16    actually following page 22 of Exhibit 46, figures 9 and 10.

17         A.    Yes.

18         Q.    That is following page 22 of the appendix, so we 

19    are now into the figures that follow the appendix that comes 

20    after the main body of the text of Defendant's Exhibit 46.

21               Do you see figures 9 and 10, Professor Wachter?

22         A.    Yes.

23         Q.    Those figures show that distributive accuracy 

24    improves in figure 10 over figure 9, is that right?

25         A.    Yes.
                                                              2222

 1         Q.    Figure 10 gives us 63 percent --

 2         A.    Yes, it does.  There is a small enhancement.

 3         Q.    And figure 10 was done with Fit 3, is that right?

 4         A.    Right.

 5         Q.    Fit 3 does not involve perfect homogeneity, does 

 6    it?

 7         A.    That's right.

 8         Q.    Figure 9 reflects Fit 1, is that right?

 9         A.    That's right.

10         Q.    So in figure 9 you are modeling perfect 

11    homogeneity, is that right?

12         A.    That's right.

13         Q.    And then when you moved modeling heterogeneity in 

14    figure 10, distributive accuracy increases, is that right?

15         A.    Yes.  This forms an interesting exception to the 

16    general rule, and I comment on that in my paper.

17         Q.    Indeed, this casts some doubt on the assumption 

18    that one needs perfect homogeneity in order to achieve the 

19    maximum improvement from adjustment, does it not?

20         A.    Well, if one made that assumption, this would 

21    show there are exceptions, certainly.  It's one of the 

22    interesting outcomes.  

23               THE COURT:  I take it you are not going to finish 

24    today, Mr. Goldin.  

25               MR. GOLDIN:  That's correct, your Honor.  I 
                                                              2223

 1    thought we had gotten to a highlight at this point.  

 2               THE COURT:  Yes, a brilliant coupe.

 3               Might this be the appropriate point to recess for 

 4    the day?  

 5               MR. GOLDIN:  From my point of view, surely, your 

 6    Honor.  

 7               THE COURT:  And on another high note.

 8               All right.  We will resume tomorrow morning at 

 9    9:30.  

10               MR. GOLDIN:  Thank you, your Honor.  

11               THE COURT:  Thank you.

12               (Adjournment taken to Wednesday, May 27, 1992, at 

13    9:30 o'clock a.m.) 

14    

15

16

17

18

19

20

21

22

23

24

25
