                                                              554

 1    UNITED STATES DISTRICT COURT
      EASTERN DISTRICT OF NEW YORK
 2    ------------------------------x
      THE CITY OF NEW YORK, et al.,
 3    
                         Plaintiffs,          88 Civ. 3474 (JMcL)
 4    
                 v.                           
 5    
      UNITED STATES DEPARTMENT OF COMMERCE,
 6    et al., 
      
 7                       Defendants.
      ---------------------------------x
 8    CITY OF ATLANTA, et al.,
                     
 9                       Plaintiffs,
      
10                v.                          May 14, 1992
                                              9:30 a.m.
11    MOSBACHER, et al.,
      
12                       Defendants.
      ---------------------------------x
13    FLORIDA HOUSE OF REPRESENTATIVES, 
      et al.
14    
                         Plaintiffs,
15                v.                          __Civ.___(JMcL)
      
16    BARBARA H. FRANKLIN, Secretary of
      Commerce,
17    
                         Defendant.
18    --------------------------------x
      Before:
19    
                 HON. JOSEPH M. McLAUGHLIN,
20    
                                              Circuit Judge
21    
      
22    
      
23    
      
24    
      
25    
                                                              555

 1    KIRK M. WOLTER,                  resumed.  

 2               THE COURT:  Mr. Solomon, you may resume.  

 3    DIRECT EXAMINATION (Continued) 

 4    BY MR. SOLOMON:

 5         Q.    Dr. Wolter, when we left off yesterday, we were 

 6    discussing the history of the use of sampling by the Census 

 7    Bureau.  

 8               You had mentioned a number of books and articles, 

 9    pieces written and you had mentioned Dr. Eckler.

10               Do you know whether Dr. Eckler is the president 

11    of the American Statistical Association?

12         A.    At one time he was, yes.

13         Q.    Do you know a book, a work by Eckler that 
                                                           

14    discusses the history of sampling in the Bureau, in 

15    particular within the decennial census?

16         A.    I have seen such a book, yes.

17         Q.    Let me ask you, please, to turn to tab 8 of the 

18    binder in front of you.  There is a excerpt of what we have 

19    marked as Plaintiff's Exhibit 704 for identification.  It is 

20    a book entitled, "The Bureau of the Census" by A. Ross 

21    Eckler.

22               Is that a work that describes or identifies the 

23    history of sampling at the Census Bureau in the forties, 

24    fifties and sixties?

25         A.    Yes, sir, it does.
                                                              556

 1         Q.    Let me ask you, please, to look at page 98 of 

 2    that work, and I would like you to focus on the paragraph at 

 3    the bottom of the page, the last full paragraph.  In part it 

 4    reads, "Those with a blind faith in the accuracy of complete 

 5    counts furnished by the censuses were naturally concerned 

 6    about the substitution of data subject to sampling error.  

 7    The way for the fullest utilization of the sampling approach 

 8    was not really opened until research revealed that 

 9    nonsampling errors may often be larger than sampling errors.  

10    This finding led to a realization that the overall accuracy 

11    of a census would be improved by taking the savings due to 

12    sampling and using them to reduce the errors of response." 

13               This work was written in 1972.

14               At the time you were at the Bureau, did you agree 

15    with that proposition?

16         A.    Oh, absolutely.  That proposition is a given, 

17    it's axiomatic in survey sampling, not only at the Bureau, 

18    but throughout the world.

19         Q.    Could you explain to the court what Dr. Eckler 

20    and you are saying?  

21               MR. MILLET:  I will object to asking the witness 

22    to speculate to another person's meaning.  

23               THE COURT:  Say what you meant, not what your 

24    co-author meant.

25         A.    I'll say what I meant.
                                                              557

 1               Statisticians generally divide all imperfections 

 2    or flaws in a census or survey into two general categories:

 3               One is called sampling error, the other one is 

 4    called nonsampling error.

 5               Sampling error simply refers to the fact that 

 6    some error or flaw comes about because you are looking at 

 7    only a part of a population and you are projecting that up 

 8    to represent the entire population.

 9               Nonsampling error comes about for all other 

10    reasons, such as if the respondent gives the wrong 

11    information in the first place or if the interviewer records 

12    the information incorrectly or if it's converted to computer 

13    readable form incorrectly and so on and so forth.

14               All those other errors or flaws or imperfections 

15    in a census or survey are called nonsampling error.

16               And, of course, this is related very much to the 

17    post-enumeration survey that we deal with here.  It is 

18    precisely because post-enumeration survey is a small well 

19    controlled survey that it is able to achieve very low levels 

20    of nonsampling error, lower than in the census, higher 

21    levels of sampling error than in the census, but on balance, 

22    because nonsampling error tends to dominate, the PES has 

23    lower total error than does the original enumeration.

24               That's the point that I made and do make.

25         Q.    Are there any other characteristics that the PES 
                                                              558

 1    as a sample survey shares with other surveys?

 2         A.    Well, thousands of characteristics with other 

 3    surveys.  It's a random sample, it utilizes conventional 

 4    estimation techniques and so on and so forth.

 5         Q.    Is sampling variability a random variability, a 

 6    random error?

 7         A.    Yes.

 8         Q.    Could you explain to the judge what a 

 9    statistician means as a random error in a survey?

10         A.    Yes, sure, I will be happy to.

11               If you think about it in the PES in question 

12    here, we have something like 5 million blocks in the country 

13    and the PES is looking at 5,000 of them, and there are 

14    thousands and thousands of possible samples of 5,000 blocks.  

15    You can just imagine that.  

16               THE COURT:  Samples of 5,000 or 5 million? 

17               THE WITNESS:  Of 5,000 blocks.

18               There are thousands and thousands of possible 

19    samples of 5,000 blocks.  Each one of them is going to give 

20    slightly different results, slightly different results, and 

21    that's what's called sampling error.

22               If the sampling is done properly, scientifically 

23    in a random way and, indeed, it was in this case, then this 

24    sampling error has the property that it's unbiased.  

25               When you average up all of these tiny little 
                                                              559

 1    errors, the average error is zero, and in that sense a 

 2    statistician would say that the survey or the sampling 

 3    design or the estimators used in the survey are unbiased.

 4         Q.    You mentioned the term bias.

 5               Is the differential undercount that results from 

 6    the uncorrected census a form of bias to a statistician?

 7         A.    Yes.  That is a nonsampling bias.  It is a 

 8    failure to count everyone.  It has nothing to do with 

 9    sampling per se, but it is a systematic differential error.

10         Q.    Is that a definition that you would use for the 

11    term bias?

12         A.    Yes.  

13               THE COURT:  Say it again, a systematic -- 

14               THE WITNESS:  It is simply a systematic error 

15    that in this case has nothing to do with a sampling process, 

16    and it has everything to do with the way in which the census 

17    methods, the enumeration methods interact with the American 

18    population.

19         Q.    Are you aware of any challenges, recent 

20    challenges to the theory underlying sample survey?

21         A.    No.

22         Q.    You mentioned yesterday in one of your answers 

23    that the Bureau had considered and throughout the decade had 

24    rejected a number of other approaches to the problem of 

25    ameliorating the differential undercount than the PES.  
                                                              560

 1    Could you for the completeness of the record go through them 

 2    quickly for the Judge?

 3         A.    Sure, I would be happy to.

 4               There are a number of different methods of 

 5    potentially estimating the undercount.  Early in the decade 

 6    the Census Bureau, including myself, wanted to look at all 

 7    of these methods.  We wanted to test all of them to see 

 8    which would have the best chance of success in 1990.

 9               One of the methods was called the pre-enumeration 

10    survey, very similar to the post-enumeration survey, but the 

11    survey occurs before census day, not after.

12               Another record is called the reverse record check 

13    method.  This method has actually been used for the last 

14    several censuses in Canada.

15               What that simply involves is drawing a sample 

16    from the previous census and tracing that sample of people 

17    forward in time to learn whether they were counted or not in 

18    the current census, and, of course, we learned that some 

19    people are counted and some aren't, and the relative 

20    proportions between the two allow Canada to estimate their 

21    total undercount by providence.

22               There are other potential methods based on what 

23    are known as administrative records.  These are other 

24    records that the government keeps, like social security 

25    records or Internal Revenue Service records or driver's 
                                                              561

 1    license records and so on.

 2               After conducting a number of scientific 

 3    evaluations of each of these various methods in the early 

 4    1980, the Bureau ultimately rejected all of these methods 

 5    for one reason or another and decided, quite clearly, I 

 6    should say, that the PES had the very best chance of success 

 7    in 1990.

 8         Q.    Are you familiar with the term or the concept 

 9    supercensus?

10         A.    Yes.

11         Q.    Tell the court what your understanding is of 

12    that, please?

13         A.    I can tell you what my general understanding is.

14         Q.    Okay.

15         A.    But I want to tell you right up front that there 

16    has never been a clear careful articulations of what a 

17    supercensus is.

18               This concept always failed the smell test, it 

19    always failed the advisability test.  We discussed it in 

20    general terms, but not to the point where we had a clear 

21    description of what it would be, so I will give you a 

22    general description and that's all.  Okay? 

23               And the general description is that, once again, 

24    you draw a sample of blocks or other well defined geographic 

25    areas like that.  This is a sample, now, imbedded within the 
                                                              562

 1    entire universe of blocks in the United States.  And then 

 2    the idea is that you do a 1990 style enumeration everywhere, 

 3    except in the sample of blocks, and in the sample of blocks 

 4    somehow you do it better; somehow you hire better 

 5    interviewers or you throw in more headquarters people or you 

 6    develop better address lists or you talk more extensively 

 7    with the local mail carrier or whatever and somehow you just 

 8    do it better.  You work harder and you do it better in those 

 9    5,000 or however many blocks there are in the supercensus 

10    sampling.

11               I guess the point is if there are better methods, 

12    the Census Bureau would use them everywhere.

13         Q.    Did the Census Bureau ever take seriously any 

14    suggestion that a supercensus be considered or developed?

15         A.    Did we ever consider the concept?

16         Q.    Take it seriously in light of the fact that you 

17    just testified that no one ever defined it coherently?

18         A.    Well, yes and no.  I mean, we took it seriously 

19    to the point that the board that I described yesterday, the 

20    undercount steering committee, discussed this carefully 

21    repeatedly.  

22               But on the other hand, like I said before, it 

23    just didn't pass the smell test.  It never rose to a 

24    sufficient level of credibility that we decided to commit 

25    resources to investigate it further.
                                                              563

 1         Q.    We talked yesterday, and I want to finish off the 

 2    lineage, if you will, of presentations that you were making 

 3    to the outside statistical advisors to the Census Bureau.

 4               Do you recall whether you made other 

 5    presentations in the 1985-1986 time frame to keep the 

 6    outside advisors abreast of what the Census Bureau was 

 7    doing?

 8         A.    I made tons of presentations to outside advisors 

 9    during that time frame.

10         Q.    Please look at the next tab in your book, tab 9.  

11    We have marked for identification PX 544, which is an 

12    October 1986 document entitled, "The 1990 coverage 

13    measurement program and final decisions about census 

14    correction." 

15               Can you identify that, Doctor Wolter?

16         A.    Yes.  This is another article that I wrote in 

17    October 1986, presented to the Census Bureau's standing 

18    professional advisory committees.

19         Q.    Were you the only one at the Bureau who was 

20    presenting the results of the research and written 

21    communications with these outside agencies?

22         A.    No.  I was the or one of the key spokesman so you 

23    often find my name appearing on these papers.  You often 

24    would have seen me making presentations, but other staff 

25    members of mine were joining in and making more detailed 
                                                              564

 1    presentations on very specific issues.

 2         Q.    Turn, please, to the next tab, tab 10, which is 

 3    PX 548.  That is an October 1986 paper by Howard Hogan 

 4    entitled, "Weakness of the 1980 PEP and solutions for 1990." 

 5               Could you identify that for us?

 6         A.    As I explained yesterday, Howard Hogan was the 

 7    gentleman who I hired to head up the undercount research 

 8    staff.

 9               This is an article he wrote and presented to the 

10    standing advisory committees in October 1986.

11         Q.    Were you present when this paper was read?

12         A.    Sure.  

13               MR. SOLOMON:  Your Honor, plaintiffs would offer 

14    PX 544 and 548.  

15               MR. MILLET:  No objection to 544.

16               I object to 548 on lack of proper foundation.  

17               THE COURT:  Exhibit 544 is admitted.

18               As to 548, lack of proper foundation, what is 

19    your problem?  

20               MR. MILLET:  Hearsay, your Honor.  

21               THE COURT:  Mr. Solomon?  

22               MR. SOLOMON:  The purpose of this, your Honor, is 

23    to show that there were a number of people who were 

24    communicating with the outside agencies.  

25               The second purpose is to show the Bureau was 
                                                              565

 1    receiving feedback from anyone who wanted to give it to 

 2    them, and the witness has testified that he heard this paper 

 3    delivered, which obviously is the foundation that Dr. Hogan 

 4    is one of the other people who are doing it, and for those 

 5    purposes we believe it ought to be admitted.  

 6               MR. MILLET:  If it is not being offered for its 

 7    truth, then I'm not sure it's relevant

 8               THE COURT:  I take it it is not being offered for 

 9    its truth, that is, nontruth use, its relevance there is 

10    just to prove --  I missed the point, to prove what, that 

11    other people were communicating?  

12               MR. SOLOMON:  The two purposes that I have just 

13    articulated are, one, that it wasn't just Dr. Wolter, there 

14    were lots OF people giving these presentations and 

15    communicating with the outside agencies, and the second 

16    purpose is that these advisory committee meetings were the 

17    place where people on the outside had themselves heard if 

18    they had problems with the design of the PES, and this is a 

19    long time ago.

20               I would submit in addition, your Honor, that this 

21    document is not hearsay.  Howard Hogan was an agent of the 

22    Census Bureau at the time he made these statements.  It is 

23    an admission.  

24               MR. MILLET:  Well, your Honor, as to that, I 

25    think Mr. Solomon knows that Dr. Hogan is not an officer of 
                                                              566

 1    the Census Bureau or a managing agent of the Census Bureau.  

 2    He certainly is an employee of the Census Bureau, but not 

 3    every statement of our employees is an admission against the 

 4    defendants.  

 5               THE COURT:  It is if it is within the scope of 

 6    their authority to perform the act as to which they make the 

 7    admission.  

 8               MR. MILLET:  The other point I simply wish to 

 9    raise is whether this is showing the communications coming 

10    to the Census Bureau, that is quite to the contrary, it is 

11    showing someone talking to outside people, not any 

12    communications coming back to the Bureau, so on its face it 

13    doesn't meet that criterion of Mr. Solomon's proffer.  

14               THE COURT:  I will admit 548 for the nonhearsay 

15    purposes you outlined.

16               MR. SOLOMON:  The only thing that is unclear to 

17    me, your Honor, is I am prepared to establish right now that 

18    Dr. Hogan made these statements as an authorized agent.  

19               THE COURT:  If you do that I will admit it for 

20    its hearsay purposes as well.  

21    BY MR. SOLOMON:

22         Q.    Dr. Wolter, was the subject matter that Dr. Hogan 

23    was here speaking to within the scope of his authority at 

24    the Census Bureau?

25         A.    Sure.  I would also add he was an employee of 
                                                              567

 1    mine and I reviewed and approved this paper, amongst many 

 2    others.

 3         Q.    Thank you.  

 4               MR. SOLOMON:  We would offer Plaintiff's Exhibit 

 5    548, everything in it.  

 6               THE COURT:  You are saying, in effect, that you 

 7    adopted everything that is contained therein when you say 

 8    you read and approved it? 

 9               THE WITNESS:  Yes, sir.  

10               THE COURT:  All right.  I will admit it as an 

11    admission as well.

12               (Plaintiff's Exhibits 544 and 548, respectively, 

13    marked for identification were received in evidence.) 

14    BY MR. SOLOMON:

15         Q.    During the time that the Bureau was evaluating 

16    and examining the design and theory underlying the PES, did 

17    it gain an understanding of the structure or the taxonomy of 

18    errors that could arise in a PES survey?

19         A.    Yes, sir.

20         Q.    Would you describe to the court what study and 

21    analysis led to that understanding?

22         A.    This was one of the major outgroths of the 1980 

23    post-enumeration program, which is here called the PEP or 

24    post-enumeration program, and as I had said yesterday, 

25    post-enumeration surveys have been around for decades, but 
                                                              568

 1    really for the first time in 1980 we focused intensively on 

 2    what could go right and what could go wrong in a 

 3    post-enumeration survey.

 4               We worked very hard in the early years of the 

 5    decade to do that.  We dug in and analyzed all of the 

 6    information in that 1980 post-enumeration survey carefully 

 7    and identified what were the big errors, what were the small 

 8    errors, and from that, by the 84-85 time period, we were 

 9    able to distill out of our learning a structure of error.  

10    We were able to, we were able to describe all known flaws in 

11    a post-enumeration survey and divide them into mutually 

12    exclusive categories.

13               That, in fact, was done.  That, in fact, was done 

14    by me and others, like Howard Hogan.

15         Q.    I want to make sure the court is understanding 

16    what we are here talking about.

17               The PES was designed to correct errors in the 

18    census.

19         A.    Yes, sir, differential error.

20         Q.    And we are here talking about studies to detect 

21    error not in the census, but in the PES?

22         A.    That's correct.

23         Q.    Okay.  Without being falsely modest, who first 

24    figured out the taxonomy of errors in the PES?

25         A.    I did.  But I would be quick to add that Hogan 
                                                              569

 1    and various other colleagues were collaborating with me all 

 2    along the way.

 3         Q.    Tell the court why was it important to figure out 

 4    the taxonomy of the error structure of the PES?

 5         A.    Well, once again, we have this serious national 

 6    problem of differential undercount.  You would like to be 

 7    able to measure the differential undercount and add the 

 8    missed people back into the raw enumeration so as to correct 

 9    for that differential undercount.

10               The only way you can be sure of that process is 

11    if you have a good grasp of the number of people missed, so 

12    for that reason we want to know how good is the PES.

13               This follows in a long Census Bureau tradition of 

14    very careful studies, scientific evaluation studies of 

15    nonsampling error in surveys, in this case studies of 

16    nonsampling error in the PES which, in turn, is a study of 

17    the undercount error in the original enumeration.

18         Q.    Was one of the purposes of trying to ascertain 

19    the error structure of the PES to try to redesign the PES to 

20    improve it?

21         A.    Oh, absolutely.  There would be at least two 

22    purposes:

23               One is to build a proper PES design, one that was 

24    bulletproof, one that was resistant to flaws, and the second 

25    purpose for understanding error structure would be to build 
                                                              570

 1    post-production, post-PES quality checks that tell us 

 2    something about whether the PES went right or wrong.

 3         Q.    Describe for the court briefly what the text on 

 4    economy of errors included.

 5         A.    Sure.

 6               Yesterday we said that all sample surveys and 

 7    censuses involved some missing data.  That's inevitable and 

 8    the PES is no exception.  So missing data is one source of 

 9    uncertainty in a post-enumeration survey, and in the 

10    original enumeration, too, I might add.

11               Yesterday we also spoke about the matching 

12    process, that is, the process of matching people counted in 

13    the PES to people counted in the original enumeration on the 

14    basis of names, addresses and other characteristics.

15               On occasion, on a rare occasion, some incorrect 

16    decisions are made in the matching process.  That's often 

17    called matching error.  That would be the second key 

18    component of error.

19               A third key component of error would be matching 

20    errors that come about as a result of information that is 

21    reported incorrectly in the PES interview.  The outstanding 

22    case of this, I don't think it occurred much in 1990, but 

23    the outstanding case in some earlier experiences was where a 

24    person misreported where they lived on census day.

25               Now, if you misreport where you live on census 
                                                              571

 1    day in the PES interview and then if you try and match the 

 2    person to the original enumeration at that misreported 

 3    address, you may declare them to be nonenumerated in the 

 4    original enumeration when, in fact, they were enumerated 

 5    somewhere else at a different address.  So that's another 

 6    major category.

 7               A fourth potential limitation on a PES would 

 8    occur in those cases where an interviewer fails to conduct 

 9    an interview and, instead, fabricates the interview simply 

10    as a way of improperly lightning their workload.

11               Another major category of error, especially in 

12    1980, was error in the process of determining which census 

13    enumerations, which original enumeration counts were correct 

14    versus erroneous, and yesterday I spoke about erroneous 

15    enumerations which occurred because of people who are born 

16    after census day but are counted nonetheless or people who 

17    have died before census day but are counted nonetheless or 

18    people that are counted in the wrong geographic areas 

19    altogether.  These are erroneous enumerations, and sometimes 

20    the PES may fail in some small ways to adequately measure 

21    the erroneous enumerations.  That would be another major

22    category.          

23               There is another category of error that is 

24    sometimes called correlation bias, sometimes called modeling 

25    error.  It probably goes under other names, too.  Some 
                                                              572

 1    people call it heterogeneity.  All it really means is 

 2    different people within a post-stratum have different 

 3    propensities to be counted or enumerated in the original 

 4    enumeration.  And a post-stratum, if you recall, is one of 

 5    these 1,392 chunks of the country that we worked with in the 

 6    1990 PES.

 7               Random error is another major category of error.  

 8    One of the preeminent examples of that is the sampling error 

 9    that I spoke about moments ago.

10               So these are a number of the major categories of 

11    limitation on PES and census results that we developed and 

12    clarified as a result of the 1980 experience and that 

13    subsequently guided our scientific work throughout the rest 

14    of the decade.

15               (Continued on the next page) 

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              573

 1         Q.    Would you assume this in one of your answers.  I 
                                                                  

 2    want to make sure I understood you.
                                         

 3               Did the Bureau also study the balancing of gross 

 4    counts and undercounts?

 5         A.    Yes.  That's an even more -- an even more 

 6    technical issue perhaps than many of the others.  It simply 

 7    refers to the fact that the PES is being used for two 

 8    purposes.

 9               One is to measure omissions from the original 

10    enumeration, people that should have been counted and 

11    weren't.

12               And the other purpose of the PES is to measure 

13    the erroneous enumerations.  That is, people who worked on 

14    it in the original enumeration and shouldn't have been.  And 

15    these two need to balance off in an appropriate way so as to 

16    properly measure the net undercount.

17         Q.    Did the Bureau vet internally the taxonomy of 

18    errors that you thought about that you just described to the 

19    Court?

20         A.    Did we vet them within the age?

21         Q.    Yes.  

22         A.    We vetted them within the agency and outside of 

23    the agency, and they appeared in numerous papers written 

24    over the years, and they certainly, as I said before, formed 

25    the basis for much of the planning work that was done and 
                                                              574

 1    much of the understanding that we have today of the 1990 

 2    PES.

 3         Q.    All although the articulation of the error, maybe 

 4    slightly different, in substance, has anyone ever proposed 

 5    that you missed an important source of error in the 

 6    taxonomy?

 7         A.    No, not at all.

 8         Q.    Okay.

 9         A.    Later in the decade, when the sample size for the 

10    PES was reduced, there came a time when another sort of 

11    arcane technical error arose in the PES.  This is another 

12    limitation on certain kinds of survey statistics.  It's 

13    called racial bias.

14               This was studied in 1980 in connection with that 

15    PES.  It only occurs when sample sizes are less than fully 

16    large.

17               In the mid-years of the decade, we weren't 

18    concerned with that issue.  Late in the decade, when the 

19    final sample size, the PES, became known, the Census Bureau 

20    reinstated this error in its overall, shall we say, profile 

21    of PES limitations.

22         Q.    Did the Census Bureau utilize the understanding 

23    that it had gained concerning the potential sources of error 

24    to in fact redesign the PES to try to avoid them?

25         A.    We.  As I said earlier, this was the -- this was 
                                                              575

 1    a kind of intellectual guideline for us.  These are the 

 2    limitations in the PES.  Solve these limitations.  Drive 

 3    these limitations to zero and you've got a great PES.  So we 

 4    used them like a checklist.  Missing data, reduce that.  

 5    Matching error, reduce that.  And that guided our 

 6    methodological work and our subsequent testing throughout 

 7    the decade.

 8         Q.    I'd like you to turn to tab 19 just for a moment 

 9    in your book.  For the record we marked that PX 703.  It's 

10    an April 1987 paper by Dr. Wolter entitled, "Issues in 

11    Considering the Technical Feasibility of Adjustment."

12               And I would like you to look at pages 18 and 19 

13    of that document, Table 11, and describe generally for the 

14    Court what steps the Bureau took to redesign the PES in 

15    light of the understandings that you gained about its 

16    potential sources of error.  And if you need this to refresh 

17    your recollection, feel free.

18         A.    Well, this is a table, obviously, and in the left 

19    column we see something called error components, and I hope 

20    you will recognize the different categories of error that I 

21    mentioned a few moments ago, such as missing data, matching 

22    error, the reporting of census data address, fabrication, 

23    blah blah blah.

24               Anyway, in the right-hand column, you will see 

25    so-called error-resistant improvements.  These were new 
                                                              576

 1    design features that were built into the 1990 generation 

 2    PES, not present in the 1980 generation PES, that were 

 3    designed to eliminate all residual flaws in the PES.

 4               For example -- and we'll be here all day if I go 

 5    through all of this.

 6         Q.    Give the Court a few samples, if you will.

 7         A.    But in the case of missing data, for example, 

 8    this was a serious problem in the 1980 PES.  In part, 

 9    because all of the interviewing was forced to occur within 

10    one week.  And as a consequence, a lot of people weren't at 

11    home --

12               THE COURT:  This is in 1980?

13               THE WITNESS:  1980.

14         A.    (Continuing) -- a lot of people weren't home.  We 

15    couldn't collect information from them.  And, therefore, we 

16    couldn't be certain as to whether they were counted in the 

17    census or not.

18               Anyway, in -- among other things, in 1990, we 

19    expanded the interview period to at least four weeks, with 

20    repeated callbacks to the household, so as to achieve as 

21    high a cooperation rate or response rate with the public as 

22    possible.  And that worked very well.

23               There are other improvements listed there as 

24    well.

25               Let me go on to matching error.  One of the great 
                                                              577

 1    difficulties in 1980 was that the PES was based on a sample 

 2    survey that the Census Bureau uses for other purposes.  In 

 3    fact, it's the national labor force survey.  It's known as 

 4    the Current Population Survey.  This survey has certain 

 5    design features that don't make it ideal for coverage 

 6    measurement.  It's ideal for labor force measurement, but 

 7    it's not very good for measuring the coverage of a Decennial 

 8    Census.  It's not based on what we call a block sample, and, 

 9    most especially, and most problematically, if that's a word, 

10    the geography -- the geographic definitions in the Current 

11    Population Survey of 1980 were actually based on the 1970 

12    census.

13               So now you've got a sample survey used to 

14    evaluate the 1980 census, but the sample survey is based on 

15    geographic definitions from 1970, and the 1970 geographic 

16    definitions, like city boundaries, and so on, don't match 

17    the 1980 definitions.  And so the two just didn't align one 

18    another while leading to some misalignment, some matching 

19    error, some balancing error.

20               We wiped out this problem in 1990.  Just 

21    eliminated it altogether.  The 1990 PES is based on 1990 

22    geography.  It's based on a block design -- now, when I say 

23    "block," incidentally, I'm talking about a city block in an 

24    urban area like Manhattan.  In rural Pennsylvania, I'm 

25    talking about well-defined chunks of geography bounded by 
                                                              578

 1    railroads or rivers or roads or other identifiable 

 2    boundaries.  So when I say "block," that's what I'm talking 

 3    about.

 4               Moving on, quality of --

 5               THE COURT:  I think I have the impression.  

 6               MR. SOLOMON:  Okay.  Thank you.

 7         Q.    I would like you to focus, however, on 

 8    correlation bias.

 9         A.    Yes, sir.

10         Q.    You mentioned that before.  Please explain to the 

11    Court what particular steps were taken in the design of the 

12    1990 PES with respect to correlation bias.

13         A.    Well, a number of steps.  Correlation bias is a 

14    catchall phrase, and it's technical jargon; and in 1980, 

15    what it meant was two-fold.  

16               First of all, the PES in 1980 occurred at about 

17    the same time as the 1980 original enumeration did.  At 

18    least part of it.  And there is confusion in the public's 

19    mind as to whether they are cooperating with the PES or with 

20    the census, and this confusion leads to certain difficulties 

21    in the conduct of the PES.  Interviewers were tripping over 

22    one another: PES interviewers versus original enumeration 

23    interviews.  We separated the interviewing period 

24    sufficiently in 1990 so that this didn't occur in 1990.

25               Okay.  The second reason that comes under the 
                                                              579

 1    general category of correlation bias is what I earlier 

 2    called heterogeneity; that is, the tendency of different 

 3    individuals to have different propensities to be counted in 

 4    the original enumeration.

 5         Q.    Did that source of error -- was that source of 

 6    error designed around or eliminated with respect to the 1980 

 7    PES?

 8         A.    Well, the Census Bureau attacked that issue 

 9    vigorously by its so-called post-stratification scheme.  The 

10    Bureau tried to chop up the country into pieces -- that is, 

11    the 1392 pieces -- such that within one of these pieces, all 

12    individuals tended to have similar probabilities of being 

13    counted in the original enumeration.

14         Q.    When --

15         A.    That wasn't present in 1980.

16         Q.    Okay.  To the extent that any error remained with 

17    respect to correlation bias, does that error tend to 

18    understate or overstate the undercount?

19         A.    To the extent that there is some residual 

20    heterogeneity, that causes the PES estimates generally to be 

21    too low.  And, in fact, it sort of forces them to be between 

22    the original enumeration and the hypothetical true 

23    population.  So that it allows the PES to get part way -- 

24    part way there, but not all the way there, but certainly a 

25    lot more the way there than was the original enumeration.
                                                              580

 1         Q.    In your opinion, did the redesign of the PES for 

 2    1990 lead to a state of affairs where, to the extent there 

 3    is any correlation bias, its only effect is to understate 

 4    the undercount?

 5         A.    Yes, sir.  It understates the undercount.  And it 

 6    tends to understate the undercount the most where the 

 7    measured undercount itself is the highest.

 8         Q.    Can you, by the way, identify the document that 

 9    you have been looking at?  This is, for the record, PX 703.

10         A.    Well, this is another one of the articles that I 

11    wrote during the decade, this one in April of '87.  It also 

12    was presented to the standing Census Bureau advisory 

13    committees.  

14               MR. SOLOMON:  We would offer PX 703.  

15               MR. MILLET:  Objection.  It wasn't on the May 1st 

16    exhibit list.  

17               MR. SOLOMON:  So far as I'm aware, your Honor, it 

18    was an oversight and the government has had this document 

19    for a week.

20               So for as I believe --

21               THE COURT:  I heard you.  

22               MR. SOLOMON:  -- they are right, it was not.  

23    They've had it for a week.  In fact, they've had it for five 

24    years.

25               THE COURT:  Is this problem going to recur?  
                                                              581

 1               MR. MILLET:  Pardon me, your Honor?

 2               THE COURT:  Is this problem going to occur with 

 3    some frequency?  

 4               MR. MILLET:  Of exhibits not on the list?

 5               THE COURT:  Yes.  

 6               MR. MILLET:  I think we had it occur from our 

 7    first day.  There are a limited number of documents.

 8               THE COURT:  Unlimited?  

 9               MR. MILLET:  No, a limited number of documents 

10    that they served upon us at the last minute after we got 

11    their mound of 701 documents.  

12               MR. SOLOMON:  Your Honor, I think it's 6 or 7 

13    documents total.

14               THE COURT:  I'll permit it then, if it is that 

15    order of magnitude.

16               (Pause)

17               THE COURT:  730 is admitted.

18               (Exhibit PX 730 for identification was received 

19    in evidence) 

20    BY MR. SOLOMON:

21         Q.    Dr. Wolter, I want to focus for one more minute 

22    on heterogeneity, the correlation bias that you made.

23         A.    Okay.

24         Q.    When the Bureau determined that it could ensure 

25    that the correlation bias, if it existed at all in the PES, 
                                                              582

 1    went only in one direction -- that is, it only understated 

 2    the undercount -- did it publish those results?

 3         A.    I'm sorry.  Did it publish what results?

 4         Q.    Did it publish the conclusion that I believe you 

 5    testified to, that because you separated the timing of the 

 6    interviews, any correlation bias would have a tendency to 

 7    understate the undercount?

 8         A.    Oh, sure.  I think that appears in print in 

 9    numerous places, and certainly I recall discussing that in 

10    numerous meetings with these various committees.

11         Q.    Notwithstanding the efforts of the Bureau, did 

12    you -- the efforts of the Bureau in the post-stratification, 

13    which I think you described tends to reduce the correlation 

14    bias, or the heterogeneity problem, did you believe that 

15    there was some likelihood that there would still be some 

16    heterogeneity error in the PES?

17         A.    Okay, sure.  I mean, let's face it:  Every 

18    American is different, right?  We are all human beings, we 

19    are all different to one extent or another.  And even after 

20    the Census Bureau's considerable efforts to divide up the 

21    country into homogeneous chunks, there's inevitably some 

22    residual unlikeness between individuals within a 

23    poststratum.

24         Q.    And did the Census Bureau publish that fact as 

25    well?
                                                              583

 1         A.    Sure.

 2         Q.    To the extent that one were to impose a standard 

 3    for adjustment that required the elimination of all 

 4    heterogeneity bias that you are now talking about, all of 

 5    the correlation bias that you are talking about, would an 

 6    adjustment ever be possible?

 7         A.    No.  We might as well pack up all of these 

 8    notebooks and go home at that point.

 9               (Laughter) 

10         Q.    When did the Bureau first publish the fact that 

11    it could not eliminate all heterogeneity bias?

12         A.    When did the Bureau first publish that?

13         Q.    Yes, sir.

14         A.    Probably in 1950.  That's been known around the 

15    agency for all these decades.  I couldn't say.  I mean, I 

16    know it's appeared in print many times, but who is the first 

17    to say that?  I'm sure it was decades ago.

18         Q.    Okay.  Let me ask you please to turn to tab 12 of 

19    your binder.

20         A.    Tab 12?

21         Q.    12, yes.  It's somewhat back in order now.  This 

22    is PX 148.  

23               For the record it is entitled, "The Technical 

24    Feasibility of Correcting the 1990 Census" by a group of 

25    authors, beginning with Childers and ending with Wolter.
                                                              584

 1         A.    Yes, sir.

 2         Q.    What is that document?

 3         A.    It's another paper that I wrote, along with 

 4    others, obviously.

 5         Q.    Do you recall where this was published?

 6         A.    This was -- I believe this was presented at a 

 7    national meeting of the American Statistical Association.

 8               THE COURT:  Some meeting.

 9               THE WITNESS:  And I would -- 

10               MR. ZIMROTH:  That's what he says about the ABA 

11    Journal, your Honor.

12         A.    Probably, in August of 1987.

13         Q.    I'd like you to turn, please, to page 3 from the 

14    end; item 9 is at the top of the page, and it says "correla- 

15    tion bias."

16               And I would like you to read --

17         A.    Excuse me.  Where are we now?

18         Q.    Sure, three pages from the end.

19         A.    Okay, yes, right.  Correlation bias.

20         Q.    Correlation bias.  

21               And I would like to read you two statements that 

22    appear in this discussion.

23               The first appears in the first column, just 

24    before the formula down there, which says:  

25               "We use post-stratification to control 
                                                              585

 1    heterogeneity bias.  The bias is not completely eliminated 

 2    because there is inevitably some residual heterogeneity 

 3    capture probabilities within poststrata," and then you go 

 4    on. 

 5               And just before item No. 10, there is a sentence 

 6    that says -- 

 7               MR. ZIMROTH:  That's on the next page.

 8         Q.    Yes.  Do you have that?

 9         A.    Yes.

10         Q.    The last two sentences there say:  

11               "The report suggests generally that there are 

12    people with very low capture probabilities who tend to be 

13    missed by both census and PES, and, thus, that an important 

14    correlation bias may be present in TARO datum causing a 

15    downward bias in estimation population sizes.  Thus, one 

16    moves closer to truth but not all of the way." 

17         A.    Yes, sir.

18         Q.    First, tell the Judge what TARO is?

19         A.    Well, that's an acronym.  The Census Bureau loves 

20    acronyms.  In this case it stands for Test of Adjustment 

21    Related Operations.  It was a test done in connection with a 

22    1986 test census in the City of Los Angeles.

23         Q.    And we will actually ask you to describe TARO in 

24    just a moment.

25               For these purposes, are those two of the places 
                                                              586

 1    where the Bureau disclosed both its belief that there was 

 2    going to be a residual heterogeneity bias and the Bureau's 

 3    conclusion that if there were any bias, it would tend to 

 4    understate the undercount?

 5         A.    Yes, sir.

 6         Q.    Okay.  Did you read in the Secretary's discussion 

 7    of residual heterogeneity or --

 8         A.    Yes, I believe I did.

 9         Q.    Okay.  Tell the Court generally what the 

10    Secretary's thrust was.

11               MR. MILLET:  I'll object to that.  The 

12    Secretary's decision speaks for itself.  He can state his 

13    opinion --

14               THE COURT:  I'll sustain that objection.

15         Q.    Tell the Court your understanding of what the 

16    Secretary was saying concerning heterogeneity bias. 

17               MR. MILLET:  Same objection.

18               THE COURT:  I'll permit him to give his 

19    understanding.  

20               Go ahead.

21         A.    Well, on its face, the Secretary seemed to be 

22    saying, to me, that there is heterogeneity in America, and 

23    somehow this is a cancer on the PES, and somehow this 

24    entirely undermines the entire enterprise and invalidates 

25    the entire enterprise.  And, therefore, among other reasons, 
                                                              587

 1    the original enumeration is far better, and we ought not 

 2    correct the original enumeration for the very significant 

 3    differential undercount in that original enumeration.

 4               And here again he's -- he wasn't presented with 

 5    the right facts, he didn't read the right facts, he didn't 

 6    hear of the right facts.  He heard, but he didn't interpret 

 7    them correctly, or whatever.  But he was led to the wrong 

 8    conclusion for whatever reason.  

 9               MR. MILLET:  Your Honor, I must renew my 

10    objection and move to strike that testimony.  It's in the 

11    Secretary's mind now.

12               THE COURT:  I will strike the last remarks about 

13    what he was told, what he heard, or must have listened to, 

14    et cetera, as speculation.

15               THE WITNESS:  Fair enough.  

16               MR. SOLOMON:  I'll be perfectly happy to have the 

17    government call the Secretary, your Honor. 

18               THE COURT:  Call him what?  

19               MR. SOLOMON:  He can call him what I call the 

20    Secretary.

21    BY MR. SOLOMON:

22         Q.    Dr. Wolter, what is your opinion concerning the 

23    Secretary's use of heterogeneity bias?

24         A.    Well, I think it's extremely unfortunate for the 

25    country and for all those who use Census Bureau data that 
                                                              588

 1    this issue is so badly misunderstood and evidently played 

 2    the role that it did in the decision that was made.

 3         Q.    Under the Secretary's reasoning, as you 

 4    articulated your understanding of it, will there ever be an 

 5    adjustment?  

 6               MR. MILLET:  I'll object to that.  That does call 

 7    for speculation, your Honor.

 8               THE COURT:  Overruled.  You may answer it.

 9         A.    Well, as we said before, every American is 

10    different.  That's going to be true ten years from now and 

11    ten years after that and ten years after that.  And if we 

12    are going to require that all Americans behave the same, 

13    then we can give up on improving the original enumeration.

14         Q.    Thank you.  

15               Let me ask you another question concerning this 

16    article.  The last page has a footnote which reads:  

17               "This paper reports the general results of 

18    research undertaken by Census Bureau staff.  The views 

19    expressed are attributed to the authors and do not 

20    necessarily reflect those of the Census Bureau."

21               Do you see that?

22         A.    Yes, sir.

23         Q.    Was it a practice at the Bureau to put that on 

24    all publications published by Bureau personnel?

25         A.    Yes, that was a policy at the Census Bureau.
                                                              589

 1         Q.    Was there a practice or procedure within the 

 2    Bureau whereby publications of Bureau personnel were 

 3    reviewed prior to the time that they were published?

 4         A.    Yes, sir.

 5         Q.    Describe for the Court generally what that 

 6    procedure was.

 7         A.    There were two components to the review of all 

 8    articles of this kind.

 9               There was a policy review, and there was a 

10    technical review.  A technical review by peers, policy 

11    review by the authors, supervisory chain.  Papers were 

12    approved on both grounds before they were presented.

13         Q.    In some sense, having gone through that process, 

14    were these papers authorized by the Bureau -- not statements 

15    by the Bureau, but authorized by the Bureau?

16         A.    What do you mean by "authorized"?  People were 

17    authorized to attend the meeting in the first place.  The 

18    Bureau didn't have to let them go to the meeting.  And 

19    people were authorized to present the paper and to publish 

20    it, and in the sense that paper passed the policy review and 

21    a technical review, and the persons were not prevented from 

22    attending.  

23               MR. SOLOMON:  The plaintiffs would offer PX 148.  

24               MR. MILLET:  I wasn't going to object until that 

25    last line.  Mr. Solomon prefaced his questions by saying 
                                                              590

 1    these were not statements of the Census Bureau, although 

 2    they were reviewed.  So on that basis, I'd have to object on 

 3    hearsay grounds.  

 4    BY MR. SOLOMON:

 5         Q.    Dr. Wolter, are you one of the authors of this 

 6    paper?

 7         A.    Yes, sir.

 8         Q.    Was this paper delivered at the 1987 proceedings 

 9    of the social statistics section of the ASA?

10         A.    Yes, sir.

11         Q.    And did it incorporate the conclusions that you 

12    had reached along with others prior to the time that you 

13    wrote it?

14         A.    Yes.

15         Q.    And did it pass the policy and technical 

16    review --

17         A.    Yes, sir.

18         Q.    -- procedures prior to the time that it was 

19    published?

20         A.    Sure.  

21               MR. SOLOMON:  We offer it.

22               THE COURT:  I will admit Plaintiff's Exhibit 148 

23    and will give his disclaimer footnote appropriate probative 

24    force on its weight.

25               (Exhibit PX 148 for identification was received 
                                                              591

 1    in evidence) 

 2    BY MR. SOLOMON:

 3         Q.    We were talking about the sources of error that 

 4    were identified in the PES as a means for redesigning the 

 5    PES and then testing the PES.

 6         A.    Yes.

 7         Q.    Did there come a time at the Bureau when the 

 8    Bureau endeavored to develop a technique for measuring the 

 9    total amount of error that might be in the PES as opposed to 

10    the total amount of error that's in the census?

11         A.    Yes, there came a time when we did that.

12         Q.    Okay.  Could you describe for the Court briefly 

13    what the history of that endeavor was?

14         A.    Well, again, in some ways, this is a real 

15    outgrowth of the 1980 experience.

16               Again, you see this evolutionary building of 

17    information over the decades.  In 1980, we didn't have the 

18    taxonomy of errors worked out in this detail, in this way.  

19    We didn't understand them quite as well as we do today.           

20    And our knowledge about each of the errors was based upon 

21    informal evidence and anecdotal evidence.  Powerful, but not 

22    the same as a systematic scientific approach.

23               And so as we moved into the decade of the '80s, 

24    we not only built on 1980 by developing the taxonomy of 

25    error in the first place, but then within each of the 
                                                              592

 1    categories of error, within the taxonomy, we built quality 

 2    checks: sound, scientific, statistically-designed quality 

 3    checks, checks to see that, Did this error occur?  Did this 

 4    error occur?  If it did, how big is it?

 5               And a first generation of these quality checks 

 6    was implemented in this TARO, that I referred to a moment 

 7    ago, in Los Angeles.

 8               I should say now, just to be clear in everyone's 

 9    mind, that there's a production PES -- that is, you produce 

10    estimates of the undercount based on a production of the 

11    PES; and then, following that, at a later point in time, 

12    there are these post-production quality checks within each 

13    of the components of error in the taxonomy.

14         Q.    Are there quality checks within the production of 

15    the PES?

16         A.    There are many.  In fact, they're massive and 

17    extensive.

18               I simply wanted to say that even after those mass 

19    of extensive quality checks within production PES, you then 

20    have a further and final check after the PES has been done.

21         Q.    And the total error analysis or the summing up of 

22    the errors that we're now talking about is the post- 

23    production analysis, is that right?

24         A.    That's right.  That's the term I usually use, 

25    post-production.
                                                              593

 1         Q.    Were papers written within the Bureau trying to 

 2    analyze the total error model that the Bureau had come up 

 3    with?

 4         A.    Yes.  Again, this was a major initiative 

 5    throughout the decade.

 6         Q.    I would like you to look, please, at first tab 14 

 7    and then tab 13.  

 8               MR. SOLOMON:  For the record, we have marked for 

 9    identification as PX 663 a document that has a draft stamped 

10    in the top right-hand corner.  It's by Mary Mulry and Bruce 

11    Spencer, called, "Total Error in PES Estimates of 

12    Population: The Dress Rehearsal Census of 1988."  

13               And we have also marked as PX 661 for 

14    identification a paper by Mary Mulry and Bruce Spencer 

15    called, "Total Error in PES Estimates of Population."  

16               What we have here is a reprint of the Journal of 

17    American Statistical Association, December of 1991.

18    BY MR. SOLOMON:

19         Q.    Can you identify those documents?

20         A.    Yes?

21         A.    What are they?

22         A.    Well, they are papers written by Mary Mulry who 

23    is a member of the Undercount Research Staff.

24               THE COURT:  Does she work for Hoaglin?

25               THE WITNESS:  She works for Hoaglin, yes, sir.  
                                                              594

 1    And Bruce Spencer was a consultant with the Census Bureau.  

 2    He is a Professor of Statistics at the Northwestern 

 3    University in Evanston, Illinois.

 4         Q.    Do these papers present the considered view 

 5    concerning the total error model that you had developed at 

 6    the Bureau?

 7         A.    Yes, sir.

 8         Q.    At the time that you -- withdrawn.

 9               Have you seen PX 663 before?  That's the draft.

10         A.    Sure.

11         Q.    Okay.  And had you read that document?

12         A.    Sure.

13         Q.    And you've seen PX 661?

14         A.    That's the previous one?

15         Q.    That's the previous one, yes, sir.

16         A.    Yes, sir.  I should say these were done in 

17    connection with what's called "The 1988 Dress Rehearsal 

18    Census."  There was a first generation of this work done on 

19    the 1987 data.  This might be viewed as a second generation 

20    analysis of total error in the PES.

21         Q.    Did the PX 663, the draft -- is that a document 

22    that you reviewed prior to the time you rendered your 

23    recommendation to the Secretary?

24         A.    Sure.  This was presented to the panel, as I 

25    recall.
                                                              595

 1         Q.    Did you see a final version of that paper prior 

 2    to the time that you made your recommendation to the 

 3    Secretary?

 4         A.    Did I see the final version of the paper?

 5         Q.    Prior to the time that you made your 

 6    recommendation to the Secretary -- the reprint that I have 

 7    here has a December 1991 date on page 839.

 8         A.    Well, probably not.  My recommendation was in 

 9    what, June of '91, so I wouldn't have seen this particular 

10    reprint at that time, but I would have seen the draft, sure.  

11               MR. SOLOMON:  Plaintiffs offer PX 663 and PX 661.  

12               MR. MILLET:  Hearsay to both and relevance to 661 

13    since he testified he didn't see it before his recommenda- 

14    tion.

15               THE COURT:  Take the relevance point first, Mr. 

16    Solomon.  

17               MR. SOLOMON:  Just of PX 661?  

18               MR. ZIMROTH:  663 was --

19               THE COURT:  The final one, yes.  

20               MR. MILLET:  661 is the final, your Honor, and 

21    that's my relevance objection.

22               THE COURT:  Okay.  I've got the wrong number, you 

23    are right.  

24               661, final.  He said that's irrelevant because 

25    the witness didn't see it on time.  
                                                              596

 1               MR. SOLOMON:  We are going to withdraw the offer 

 2    of that document.

 3               THE COURT:  So we are down to 663, which is the 

 4    draft.  I'll admit it, 663 is admitted.

 5               (Exhibit PX 663 for identification was received 

 6    in evidence) 

 7    BY MR. SOLOMON:

 8         Q.    Dr. Wolter, did anyone bring to your attention at 

 9    any time any evidence that the total error model that the 

10    Bureau created was incomplete or inaccurate in any respect?

11         A.    No.

12         Q.    Are you familiar with the concept of "loss 

13    functions"?

14         A.    Excuse me?

15         Q.    Loss functions.

16         A.    Yes, sir.

17               THE COURT:  What about them?

18               THE WITNESS:  What about them?

19         Q.    Are you familiar with the term "loss functions"? 

20    I'm sorry.

21         A.    Okay, sure.

22         Q.    Okay.  Tell the Court briefly what they are as it 

23    relates to the PES.

24         A.    Well, once again, the problem here is 

25    differential undercount, and we'd like to do better than the 
                                                              597

 1    original enumeration.

 2               The issue is:  What does "better" mean?  How do 

 3    you define what better is?  How do you know that a set of 

 4    corrected data is better than a set of original raw data?           

 5    What does better mean anyway?  

 6               And "loss functions" are simply mathematical, 

 7    statistical expressions of what "better" means.  They 

 8    summarize residual error in a set of data, in a set of 

 9    information.

10               In this case, you might have a loss function for 

11    the original enumeration, summing up the -- summing up the 

12    error in that data, and the same loss function for the 

13    corrected data, summing up the error in that data, or that 

14    information.

15         Q.    Did the Bureau use loss function analyses to try 

16    to ascertain the quality of the 1990 PES?

17         A.    Yes, it did.

18         Q.    All right.  When did you first start thinking 

19    about the use of loss functions for analyzing the PES at the 

20    Bureau?

21         A.    Perhaps in 1983 or before.  I don't recall 

22    exactly.

23         Q.    Was there analysis in the evaluation at the 

24    Bureau of the issues of what is better and the other issues 

25    that you mentioned throughout the course of the '80s?
                                                              598

 1         A.    Okay, sure.  This was, again, a major topic of 

 2    discussion within the Bureau.  As we indicated yesterday, I 

 3    personally, and the Bureau generally, tried to engage the 

 4    professional community, and other communities of interested 

 5    census users, in discussing appropriate loss functions, or, 

 6    in laymen's language, discussing appropriate definitions of 

 7    what "better" means anyway.

 8               And there was paper -- there were papers written 

 9    on this.  There was much discussion of this.  There was a 

10    narrowing of opinion on this, and so forth.

11         Q.    As a result of the narrowing of the opinion, did 

12    you ever conclude that a single loss function was the right 

13    one to use to test the PES?

14         A.    No, sir.

15         Q.    What did you conclude?

16         A.    It's my general belief that the decennial census 

17    is a large massive complex operation, and ultimately a large 

18    complex set of data.  In my view, no one measure is adequate 

19    to describe the error in that data, no one measure is 

20    adequate to describe what "better" means.

21               I much prefer to consider a range of definitions 

22    of what "better" means.  And I might add, if I might, that 

23    this follows a Census Bureau tradition.  The Bureau, from 

24    the late '60s and in through the '70s, always considered and 

25    used a range of three or four or five dimensions of "better" 
                                                              599

 1    in evaluating information for small areas.

 2         Q.    In your opinion, are there any measures of 

 3    "better" when it comes to the measurement of the loss in the 

 4    PES versus the loss in the census that are unreasonable?

 5         A.    Okay, sure.  One could -- an uncareful person 

 6    could always make arbitrary and bizarre definitions of what 

 7    "better" means that would lead to silly results.

 8               I'm talking about reasonable approaches here.

 9         Q.    Look.  I want to know whether, when you were at 

10    the Bureau, in thinking about this question, you considered 

11    whether particular loss functions, if used to evaluate the 

12    census and the PES, would be unreasonable?

13         A.    Did we consider it?  Sure.  Our advisors 

14    considered it.

15         Q.    Let me ask you --

16         A.    I think among other -- among other issues, we 

17    felt generally through the decade, our advisors felt 

18    generally that one should -- one should evaluate the size of 

19    an error in a census count in relationship to the size of 

20    the place.

21               In other words, from a national prospective, a 1 

22    percent error in counting Nevada is far less serious than a 

23    1 percent error in counting the State of New York or the 

24    State of California, or the State of Alabama or Georgia.

25               Now, to Nevada, 1 percent is important, but from 
                                                              600

 1    a national perspective, which is the only perspective the 

 2    Census Bureau can possibly take and does indeed take, one 

 3    must view inaccuracy in a set of data in relationship to 

 4    size of place, and that was a strong feeling that we had and 

 5    that our advisors had about what "better" means and in turn 

 6    about what loss functions should be utilized.

 7               MR. SOLOMON:  Two short questions.  And then if 

 8    your Honor wishes to take a break, we'd be happy to.

 9               THE COURT:  Sure.

10         Q.    Dr. Wolter, look at tab 7, which is the NAS 

11    panel, the Bicentennial Census, which we marked and used 

12    yesterday as PX 2.  Page 282.

13               In recommendation 7.2, I'm just going to read the 

14    tail end:  It says: 

15               "Thus, we recommend against loss functions based 

16    solely on the number of political entities losing or gaining 

17    through adjustment."

18               Is that a reflection of one of your advisory 

19    committees agreeing with the statement that you just 

20    expressed?

21         A.    Sure.  Absolutely.

22         Q.    Did you see the Secretary's decision of loss 

23    functions?

24         A.    Yes, I did.

25         Q.    And can you describe to the Court your 
                                                              601

 1    understanding of which one he spoke of?

 2         A.    There was a passage where he spoke of the number 

 3    of states that were improved under corrected data versus the 

 4    number not improved under corrected data.

 5         Q.    Is that what you were just talking about, and is 

 6    that what the NAS is talking about as having recommended 

 7    against using?

 8         A.    Yes, this was recommended against.

 9               Now, but don't push me too far on that.  I'm not 

10    saying the one -- the one shouldn't look at the number of 

11    states, they are improved or not improved.  I think it's 

12    perfectly reasonable to do that, but if that's the only 

13    thing you do, then that is unreasonable.  

14               MR. SOLOMON:  Thank you.

15               THE COURT:  All right.  Let's take a 15-minute 

16    break, ladies and gentlemen.

17               (Recess)

18               (In open court) 

19    KIRK WOLTER, resumed

20               THE COURT:  Ready?  

21               MR. SOLOMON:  Thank you.  

22    DIRECT EXAMINATION CONTINUED

23    BY MR. SOLOMON:

24         Q.    Dr. Wolter, you made reference a number of times 

25    to the field tests of the PES design.
                                                              602

 1         A.    Yes, sir.

 2         Q.    I want to turn to that.

 3               What field testing took place?  Would you simply 

 4    list where they are first, and then we'll go through them 

 5    quickly for the Court.

 6         A.    Sure.  There were tests in Tampa, Florida, in 

 7    1985.  There were tests in Los Angeles, California, in 1986.  

 8    There were tests in Mississippi in 1986.  There were tests 

 9    in North Dakota in 1987.  There were tests in St. Louis, in 

10    Missouri and in Washington State in 1988.

11         Q.    And all of these were tests of the PES?

12         A.    They were tests of the original enumeration and 

13    of the PES.  And then there are a few other more specialized 

14    tests as well, but principally they are tests of the 

15    original enumeration and of the post-enumeration survey.

16         Q.    Why were these field tests conducted?

17         A.    Well, for several purposes.  Earlier we described 

18    what went right and what went wrong in 1980.  We described 

19    improved error-resistant designs.  We described this logical 

20    development, and so forth.  But the census just doesn't stop 

21    at that point.  The Census Bureau, as a matter of its 

22    history and corporate culture doesn't simply say:  Well, 

23    this is going to work, that's it, this is what we'll do.  

24    Instead, of the Census Bureau carefully field-tests all of 

25    its procedures and demonstrates that everything works 
                                                              603

 1    properly.  That's true not only of the original enumeration 

 2    but it's true, in particular, of the PES in this case.

 3         Q.    Were the tests used to assist in the redesign in 

 4    the PES?

 5         A.    Sure.  There's an iterative process under way.

 6               THE COURT:  I dent hear you.  What process?       

 7               THE WITNESS:  There is an iterative process 

 8    whereby we learn from 1980, we make improvements.  We test 

 9    those improvements.  We evaluate the tests.  We learn and 

10    make further improvements.  We test the further 

11    improvements.

12               We evaluate tests of further improvements, and we 

13    learn and test and learn and test, and back and forth.  And 

14    that went on from '85 to '86 to '87 to '88.

15         Q.    Did the Bureau advise its outside advisors of the 

16    fact that it was going to be doing these tests and of the 

17    results of the tests?

18         A.    In every step along the way.  And advised the 

19    Congress and all other parties to the overall census 

20    processes of all these tests.

21         Q.    I'd like you to look at the next tab, tab 15, 

22    very briefly.

23         A.    15?

24         Q.    15, just for the record is PX 542.  

25               It is a November 1985 document entitled:  "1990 
                                                              604

 1    Census Adjustment Research Progress and planning."

 2               Can you identify that document?

 3         A.    Yes, sir.  That's another article that I wrote.

 4         Q.    Is one of the purposes of this article to bring 

 5    the outside advisors up to date on the testing that had been 

 6    done by then and the subsequent tests that they were 

 7    planning?

 8         A.    Yes, sir.  

 9               MR. SOLOMON:  We would offer PX 542.  

10               MR. MILLET:  No objection.

11               THE COURT:  542 is admitted.

12               (Exhibit PX 542 for identification was received 

13    in evidence)

14               (Continued on next page) 

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              605

 1    BY MR. SOLOMON:

 2         Q.    Tell the court about the Tampa test in 1985.

 3         A.    Surely.

 4               Two of the very critical limitations on the 1980 

 5    PES was missing data in that PES and I spoke about that 

 6    earlier, and secondly there was some difficulty with 

 7    matching in that test or in that PES, and so very early in 

 8    the decade, as early as 85, and even before, we began 

 9    vigorous work to try to eliminate these components of error 

10    from a PES.  We mounted a test PES in Tampa with two 

11    objectives:

12               One objective was to see whether we could drive 

13    down the missing data rates and solve that problem.  The 

14    other purpose of the 85 test was to evaluate the efficacy of 

15    a new automated matching process that we had developed.

16               Now, I spoke about that earlier.  It's an expert 
                                                                  

17    system, it's automated, it is designed to skim the cream, 

18    that is, to match most of the easy cases from a PES to the 

19    original enumeration, and as I said, testing that new 

20    automated expert system was one of the --  was the second 

21    key purpose of the Tampa test.

22         Q.    What were the result of the tests?

23         A.    Well, the results were very good.  We drove down 

24    the missing data rate to much lower levels than had been 

25    experienced in 1980 and, secondly, the new automated 
                                                              606

 1    matchers proved to be very successful.

 2               It, first of all, matched 50, 60 or more percent 

 3    of PES people and, secondly, it did so accurately, so that 

 4    encouraged us to carry further with that process.

 5         Q.    In the earlier document that we saw where the 

 6    innovations with the corrections, the redesign were listed 

 7    in relation to each of the components of error, I also 

 8    thought I saw something about a new questionnaire.

 9               Was a new questionnaire tested in the 1985 Tampa 

10    test?

11         A.    Yes, it was.

12               If you recall, the 1980 PES was based on this 

13    national labor force survey, the CES, the current population 

14    survey, which has a labor force questionnaire.  It's not at 

15    all designed for measurement of undercount in the census.  

16    Nonetheless, that's what we had in 1980 and that was one of 

17    the grave difficulties in 1980.

18               So in preparation for 1985 and beyond, we set 

19    about the task of designing a questionnaire, an interviewing 

20    process designed solely to measure coverage in a decennial 

21    census, and that was tested, as I said, in 1985.

22         Q.    Were new training procedures tested in 1985 as 

23    well?

24         A.    Yes, sir.

25         Q.    And sample design of the PES itself, was the new 
                                                              607

 1    sample design also tested?

 2         A.    Yes, it was.  Again, moving away from the design 

 3    utilized --  let me stop for a second.  

 4               When I use the word "design," it is the word the 

 5    statistician uses to describe the way in which the sample is 

 6    selected.  That's all we mean by sampling design.  

 7               And the current population survey sampling design 

 8    was not an ideal design for measuring undercount in the 

 9    census.  If you recall, it's based on old geography and it 

10    has other problems to it.  

11               So the new design is based on a sample of blocks, 

12    well defined blocks, among other things, and that, that 

13    worked very well in 1985 and beyond.

14         Q.    What were the results of the test as it related 

15    to the questionnaire in the samples?

16         A.    We felt we were gathering accurate information by 

17    way of the new questionnaire, information that allowed an 

18    accurate determination as to whether a PES person was 

19    counted in the original enumeration or not.

20         Q.    Let's turn to the 1986 tests.

21               Where would you like to begin?  Mississippi?

22         A.    Sure.

23         Q.    What was the Mississippi test to test?

24         A.    The Mississippi test was not a test across the 

25    state, it was a test in some isolated rural areas of the 
                                                              608

 1    state, and the 1985 tests had been done in Tampa, which is a 

 2    city, so the Mississippi test was the first census and PES 

 3    test in a rural area.  

 4               And it's important to test these different kinds 

 5    of areas.  Why?  Well, principally because addresses are 

 6    different in a rural area.

 7               You don't have street name, house number, type 

 8    addresses in rural Mississippi like you did in Tampa, 

 9    instead you got world route this, post office block that, 

10    the corner of Highway A and B and so on, and this brings 

11    about very different issues in conducting a PES.  It brings 

12    about different geographic issues, different interviewing 

13    issues, different supervision issues, and it was important 

14    simply to begin gaining experience with the new PES in that 

15    kind of rural environment.  It was as simple as that.

16         Q.    What were the results of the Mississippi test?  

17    What did the Bureau learn as a result of the Mississippi 

18    test?

19         A.    There wasn't substantial learning from 

20    Mississippi, other than all seemed to work very well, as 

21    well as had worked in Tampa the year before.

22         Q.    The Los Angeles test in 1986, is that the TARO  

23    that you made reference to before?

24         A.    Yes, it is.

25         Q.    What was that test designed to test?
                                                              609

 1         A.    Well, you might guess from the name what it was 

 2    designed to test.  It was, it was designed to put in place 

 3    for the very first time in U.S. history, not only in PES, 

 4    which had been put in place for decades, but all of the 

 5    other apparatus, activities, tasks necessary to carry 

 6    through a census correction process.

 7               In other words, we had the correction process 

 8    worked out at that point in time, we had the blueprints, and 

 9    now it was a matter of learning, well, let's work it out 

10    from an implementation standpoint:

11               Can we do it? 

12               That's always the Census Bureau's approach and 

13    that was the approach in this particular case.

14         Q.    What kinds of things are you talking about when 

15    you say carry it through to the end?

16         A.    I'm talking about conducting the PES interview, 

17    I'm talking about conducting the matching that we have 

18    spoken about several times, the imputation for missing data 

19    that we have spoken about several times, the estimation of 

20    the size of the population that we spoke about several 

21    times, I'm speaking about the removal of sampling error that 

22    we spoke about yesterday, and then I'm talking about 

23    carrying the estimates of corrective population size from 

24    aggregate areas within Los Angeles down to local areas 

25    within Los Angeles, and then finally I'm talking about 
                                                              610

 1    post-production quality checks to see at the very end of it 

 2    all how well did you do, did anything totally unexpected 

 3    happen that might have compromised the results.

 4         Q.    The removal of sampling error that you were just 

 5    talking about, that's the smoothing that we have been 

 6    talking about, is that right?

 7         A.    Yes, sir.

 8         Q.    And that was done in the 1986 TARO test?

 9         A.    Yes, sir.

10         Q.    And the post-production test, was that the 

11    analysis of the total error with the total error model that 

12    we were just talking about?

13         A.    Yes, but, but, but.  We learn and get better and 

14    get better, and first generation quality checks and total 

15    error analyses were taking place in connection with Los 

16    Angeles, second generation came later, in 88, and third 

17    generation came later in 90.  So understand that this is 

18    evolving and proving work along the way.

19         Q.    What is a systematic observer study?

20         A.    In Los Angeles, we hired anthropologist and other 

21    field workers to work in local difficulty count 

22    neighborhoods.  This builds on work that was done, I 

23    believe, as early as the seventies.

24               The objective of this work, very simply, is for a 

25    field worker, an anthropologist to become immersed into 
                                                              611

 1    community, to sit on the doorstep and get to know the 

 2    residents, to talk with people on the street, to talk with 

 3    people in the bar, to talk with people in the grocery store 

 4    and to generally spend an extended period of time getting to 

 5    know the people and overcoming fear in the community.

 6               Now, what I'm talking about here are difficult to 

 7    count neighborhoods.  I'm talking about people who may have 

 8    reason to fear government, I'm talking about people who may 

 9    have reason to fear a census enumerator.

10               A census enumerator who comes into a neighborhood 

11    to conduct interviews is there for a relatively brief time 

12    and has to try and overcome that fear and does so with mixed 

13    success.

14               The anthropologist, on the other hand, immersed 

15    him or herself in the neighborhood and attempts over a 

16    prolonged period of time to overcome that fear.  

17               THE COURT:  How prolonged? 

18               THE WITNESS:  I'm talking about a month or two.

19         A.    And in this case the anthropologist were getting 

20    to know the undocumented workers, the illegal aliens in the 

21    neighborhood, the people living in garages and mobile homes 

22    behind the main housing unit and so forth.

23         Q.    What did the Bureau learn from the 86 tests that 

24    you are now referring to?

25         A.    This anthropological work?
                                                              612

 1         Q.    Yes.

 2         A.    Well, it was, it was, in my mind, confirmation of 

 3    the PES results.  It was in my mind confirmation of the 

 4    basic principle that I stated earlier, namely that the PES 

 5    results tend to lie between the original enumeration and the 

 6    hypothetical true population.

 7               The PES was finding these missing people, we 

 8    could see that in our PES data, and the systematic 

 9    anthropological observers were telling us the same story as 

10    the PES data was, so they were confirming one another.

11         Q.    What were the results of the 1986 TARO tests 

12    generally?

13         A.    Well, the results generally were perhaps 

14    two-fold:

15               First of all, as I said, the purpose of this test 

16    was to demonstrate operational feasibility, and it clearly 

17    did that.  And secondly, the purpose was to have a 

18    runthrough on these first generation quality checks, and 

19    they were very successful as well in and of themselves, but 

20    more to the point, they were successful in telling us that 

21    the PES was not only operationally feasible, but it was also 

22    technically feasible, that is, it seemed to have an 

23    extremely low margin of error.  Not that further improvement 

24    wasn't possible, but the quality checks told us that we were 

25    on the right track.  They told us if you do it right again 
                                                              613

 1    in 1990, you are going to be successful.

 2         Q.    Tell the court about the 1987 North Dakota test.

 3         A.    1987 was a small test, 86 was a major test, 88 

 4    was a major dress rehearsal.  87 is a small test in between 

 5    and simply didn't gather the attention that these two more 

 6    major tests did, but it was a test in another world 

 7    environment, North Dakota.

 8         Q.    One of my favorites.

 9               Did it test for fabrications?

10         A.    Yes.

11         Q.    Please tell the judge the story of that test.

12         A.    Okay.

13               One of the categories of potential PES error is 

14    the category called fabrication, that is, the possibility 

15    that a PES interviewer might fail to complete the interview 

16    and, instead, will simply make up the interview, fabricate 

17    the results.  

18               There was reason to believe that this occurred in 

19    1980.  There was reason to believe that it has occurred in 

20    some prior Census Bureau surveys.  And the question is, was 

21    it occurring, is it occurring in the 86, 88 --  excuse me --  

22    86, 87, 88 test PESs.  And to control this thing called 

23    fabrication, we layered in a massive level of quality 

24    checking within the production of the PES data itself.

25               There was massive supervisory review of the 
                                                              614

 1    interviewing process, telephone callbacks to households to 

 2    confirm that, in fact, interviewers had completed interviews 

 3    as scheduled, and in 86 our post-production quality checks 

 4    indicated that the in-production quality checks were 

 5    massively successful in eliminating or enormously reducing 

 6    fabrication.

 7               Notwithstanding that good evidence, some census 

 8    executives thought, well, is that enough?  Should we be 

 9    doing more?  Is there some possibility that we have missed 

10    some fabrication that existed and we just don't know about 

11    it for whatever reason? 

12               So a plan was hatched to fabricate some 

13    fabrications, and, indeed, that's what we did.  At 

14    headquarters we made up some fabricated interviews and 

15    submitted these fabricated interviews as part of the work 

16    flow in the North Dakota test.

17               The objective of this, of course, was to see 

18    whether our in-production quality checks identified all of 

19    these fabricated interviews and eliminated them and replaced 

20    them by corrected interviews.

21               And sure enough, every last one of these 

22    fabricated fabrications was identified and, as a 

23    consequence, I think all had renewed confidence that the 

24    in-production quality checking procedure was working to a 

25    very substantial extent, and that gave us good confidence to 
                                                              615

 1    move forward.

 2         Q.    When you say each of the fabrications was 

 3    identified, they were identified during the in-production 

 4    quality control checks, is that right?

 5         A.    Yes, that's right.

 6         Q.    Who was the principal proponent at the Bureau for 

 7    this additional test?

 8         A.    A generally by the name of Charlie Jones.  Jones 

 9    was head of the decennial census at that time.

10         Q.    Tell the court about the 1988 tests in St. Louis 

11    and in rural Missouri and Washington state.

12         A.    These are called dress rehearsal tests.  In the 

13    same way that actors conduct address rehearsal of a play, 

14    the Census Bureau conducts address rehearsal of a census.  

15    The objective is to carry through the final census plan in a 

16    test and to prove that the final census plan works.

17               These dress rehearsal tests occurred in an urban 

18    environment, that was St. Louis, with a substantial black 

19    population.

20               The test was furthermore conducted in some 

21    semirural and smaller city areas in Missouri, and finally it 

22    was conducted, another dress rehearsal test was conducted in 

23    some extremely rural areas of Washington state.

24               These were, in the first place, tests of what was 

25    to be the 1990 original enumeration and secondarily or in a 
                                                              616

 1    companion form they were tests, dress rehearsal tests of 

 2    what was to be the 1990 PES.

 3         Q.    Can you tell the court what the results were of 

 4    the PES tests in those areas?

 5         A.    Well, again, the PES was conducted as planned, 

 6    data was gathered, estimates of total population were 

 7    produced, sampling error was removed, in-production quality 

 8    checking was done, post-production quality checking was done 

 9    and then a second generation of the total error analysis was 

10    done.  

11               And I might add that the article you showed me 

12    earlier today by Mary Mulry and Bruce Spencer summarized the 

13    outcome of those post-production quality checks.

14         Q.    Just for the record --

15         A.    I might also add that that paper formed the basis 

16    for not only a paper at the American Statistical 

17    Association, but a major invited paper at that meeting.

18         Q.    The document that we are talking about is tab 14, 

19    PX 663?

20         A.    I'm sorry --  

21               THE COURT:  We are having some fun.  

22               MR. SOLOMON:  Don't let me interrupt.

23         Q.    Look at tab 14 of your book, PX 663, to make sure 

24    the record is clear.

25               That is the analysis that you are talking about, 
                                                              617

 1    correct?

 2         A.    Yes, sir.

 3         Q.    You testified before that the fact that the 

 4    Bureau was making these tests was published and we saw one 

 5    of the articles.  The results of the tests were also 

 6    published, correct?

 7         A.    Of the dress rehearsal or --

 8         Q.    Of the various tests that you have now described.

 9         A.    Oh, sure, yes.

10         Q.    Would you look, please, at tab 17, we have marked 

11    for identification PX 546.  It is an article by Hogan and 

12    Walter entitled, "Measuring accuracy on a post-enumeration 

13    survey." 

14               Can you identify that?

15         A.    Yes.  It's another article I wrote.  This is 

16    published in a journal called Survey Methodology.  It 

17    summarizes the results of the quality checking that was done 

18    of the 1986 TARO and is in some ways an outgrowth of that 

19    paper you showed me earlier by Childers, et al.  

20               MR. SOLOMON:  The plaintiffs offer 546.  

21               MR. MILLET:  No objection.  

22               THE COURT:  Exhibit 546 is admitted.  

23               (Plaintiff's Exhibit 546 marked for 

24    identification was received in evidence.)

25    BY MR. SOLOMON:
                                                              618

 1         Q.    As a result of the analysis of the TARO, was 

 2    there work published, documented within the Bureau about 

 3    those results? 

 4               Did the Bureau internally share within the Bureau 

 5    the results of the TARO analysis?

 6         A.    Well, sure.  I mean, papers and papers and reams 

 7    of material was produced and reviewed within the Bureau and 

 8    outside the Bureau.

 9         Q.    Please look at tab 18, which is a large exhibit.  

10    I hope we won't have to spend much time on it.  For 

11    identification it is PX 156.  It is a memorandum from Kirk 

12    M.. Wolter to the undercount steering committee with a large 

13    number of attachments.

14               Can you identify that for the court, please?

15         A.    Yes.  This is a, this is a document that 

16    predated, predated tab 17.  This mass of material was 

17    produced by my staff in the spring of 1987.  It was our 

18    first assembly of information about the quality checks and 

19    results of the 1986 TARO.  And then subsequently over the 

20    months and year that followed we gradually began distilling 

21    down this information in writing in a more concise way, and 

22    ultimately the information was published in Survey 

23    Methodology in the form that we spoke about a moment ago.  

24    But I want you to know that this, this inch thick mass is 

25    the very first generation of that material.
                                                              619

 1         Q.    The date of this document is April 1987.

 2               Was the publication within the Bureau of this 

 3    document in accordance with the time schedule that you had 

 4    told us about yesterday?

 5         A.    Yes, sir.

 6               If you recall yesterday I said that one of the 

 7    key decision points in the decade as established by the 

 8    board was to decide in the spring of 1987 whether the 

 9    undercount measurement processes were technically and 

10    operationally feasible and, thus, whether the Census Bureau 

11    would put in place work to carry forth this enterprise in 

12    1990 and whether the Bureau would seek funding to carry 

13    forth the work in 1990, and this mass of paper was one of 

14    the key inputs to the board in making this very important 

15    decision in the spring of 1987.

16         Q.    At the time that you circulated this to the 

17    board, had you concluded in your mind that correction of the 

18    1990 census based on a properly functioning PES was 

19    feasible?

20         A.    Yes, based on what was learned here, as well as 

21    our results in Mississippi and Tampa, as well as based on 

22    all of the decades' background preceding that, I concluded 

23    that we had substantial likelihood of success in 1990.  

24               MR. SOLOMON:  Plaintiffs offer PX 156.  

25               MR. MILLET:  I have to objection.  This document 
                                                              620

 1    has documents within documents and hearsay within hearsay 

 2    and it's all being offered for its truth.  I don't think we 

 3    have a foundation.

 4               I would, for example, I point your Honor to page 

 5    2493, which purports to be the results of an ethnographic 

 6    study that itself is hearsay, and within it contains 

 7    hearsay.  

 8               THE COURT:  It does seem to be a rather large 

 9    collection of miscellanea.  

10               MR. SOLOMON:  Your Honor, it may be hearsay, but 

11    I don't think it is miscellanea.  

12               As the witness just described, this was what was 

13    given to the undercount steering committee so that they 

14    could make their determination whether in their judgment 

15    adjustment was technically feasible.  

16               I think I am about to demonstrate through this 

17    witness that they, in fact, agreed with his recommendation 

18    and conclusion.  So I don't think it is miscellanea.  

19               THE COURT:  Maybe if you take it that far, 

20    develop from the witness that they relied on it, I will 

21    think differently; maybe.  

22               MR. SOLOMON:  Fair enough.  

23    BY MR. SOLOMON:

24         Q.    As a foundation, Dr. Wolter, were all of the 

25    papers that are attached to your memorandum created by 
                                                              621

 1    Census Bureau personnel or personnel --  or agents of the 

 2    Census Bureau?

 3         A.    Yes, sir.

 4         Q.    And did this document get circulated to the 

 5    undercount steering committee?

 6         A.    Yes, sir.  Not only did it get circulated, but as 

 7    it began to get circulated I spent a full day, on the order 

 8    of ten, twelve hours, describing this document to the 

 9    undercount steering committee at a special meeting.

10         Q.    Was there discussion about these series of papers 

11    at the meeting?

12         A.    Yes.  That was the purpose of the meeting.

13         Q.    As a result of that meeting, was there a 

14    conclusion reached by the undercount steering committee 

15    concerning the technical feasibility of adjusting the 1990 

16    census?

17         A.    Yes.  Of course, that wasn't the only meeting.  I 

18    mean, in this time frame there was meeting after meeting 

19    after meeting, reviewing this, digesting it, probing 

20    further, but ultimately as a result of that process a 

21    conclusion was reached.

22         Q.    What was the conclusion, sir?

23         A.    The conclusion was reached, and this was 

24    announced finally by the director of the Census Bureau, a 

25    gentleman by the name of Jack Keane, that the Bureau had 
                                                              622

 1    determined that it would put in place or would plan to put 

 2    in place work to carry forth this enterprise in 1990 and 

 3    that it would seek funding to do so.  

 4               MR. SOLOMON:  Plaintiffs renew their offer of PX 

 5    156.  

 6               MR. MILLET:  Defendants renew their objection. 

 7               THE COURT:  I will admit 156 much as you would 

 8    admit the minutes of a board of directors meeting, just to 

 9    establish how they functioned, what they considered, how 

10    they went about doing their business.  

11               MR. SOLOMON:  It has been called to my attention 

12    to the PX number that I have given your Honor, PX 156, is 

13    wrong, it is PX 5.  My apologies.  

14               THE COURT:  Plaintiff's Exhibit 5?  

15               MR. SOLOMON:  Yes, 5.  

16               THE COURT:  It is admitted as 5, not 156.  

17               MR. SOLOMON:  Thank you, your Honor.  

18               (Plaintiff's Exhibit 5 marked for identification 

19    was received in evidence.)

20    BY MR. SOLOMON:

21         Q.    At the time that the Bureau was considering the 

22    issue of technical feasibility and prior to the time that it 

23    reached the conclusion that you just testified to, did the 

24    outside advisors that you have been talking to for the years 

25    preceding weigh in on the subject?
                                                              623

 1         A.    Yes, sir.  

 2               MR. MILLET:  It calls for hearsay.  

 3               THE COURT:  I really don't have the question in 

 4    hand.

 5               What is the question?  

 6               MR. SOLOMON:  Whether the outside advisory 

 7    agencies communicated to the Bureau their views about 

 8    whether the Bureau should proceed with seeking the funding 

 9    and the design conducting of the PES.  

10               THE COURT:  That much certainly is not hearsay.

11               Objection overruled. 

12               THE COURT:  Did they communicate their views to 

13    you? 

14               THE WITNESS:  Oh, yes.

15         Q.    Did you consider those views in making your own 

16    conclusion?

17         A.    I, I reached my own conclusion based on the data 

18    and based on the conduct of the work.

19               It was very gratifying to me that the outside 

20    advisors reached similar conclusions.  I would have been 

21    surprised if they hadn't.  But, in fact, I had arrived at my 

22    opinion in parallel rather than as a result of.  

23               MR. MILLET:  Your Honor, I move to strike as 

24    hearsay.  

25               THE COURT:  Overruled.
                                                              624

 1         Q.    Who were the outside advisors that you were just 

 2    making reference to?

 3         A.    Well, the principal advisors, and leaving aside 

 4    secondary advisors, would be the standing advisory committee 

 5    of the American Statistical Association, the standing 

 6    advisory committee on population and the National Academy of 

 7    Sciences panel.

 8               Now, the two standing advisory committees, as I 

 9    recall, didn't have this massive document in part because of 

10    timing.

11         Q.    That is PX 5 that your referring to there?

12         A.    Yes.

13               I can't remember timing exactly, but I believe 

14    their meeting was held shortly before we released this 

15    document, and so they had available to them other shorter 

16    papers with less evidence, frankly, less in the way of 

17    numerical evidence and more in the way of descriptive 

18    evidence.

19               On the other hand, because of timing, the 

20    National Academy panel did have this lengthy document, and I 

21    believe they held a special meeting simply to review this 

22    material, and that probably occurred, I would guess, around 

23    the 10th or 15th of May, somewhere in that general area in 

24    1987.

25         Q.    Do you know a doctor Leo Breiman?
                                                              625

 1         A.    Do I know him personally?

 2         Q.    Yes.

 3         A.    Well, I know of him.

 4         Q.    During any of the meetings that we are talking 

 5    about that you attended with the outside advisors or any of 

 6    the internal meetings at the Bureau leading up to the 

 7    Bureau's decision in the spring of 1987, did Dr. Breiman 

 8    attend any of those?

 9         A.    No.

10         Q.    Did he ever communicate with you prior to that 

11    time concerning any of the issues of the PES?

12         A.    No, sir.

13         Q.    Since that time, while you were at the Bureau, 

14    did you communicate with him or did he communicate with you?

15         A.    To my knowledge, Leo Breiman was never at the 

16    Census Bureau, never commented on Census Bureau programs, 

17    was never familiar with the surveys and censuses conducted 

18    at the Census Bureau.  But then I do recall --  

19               MR. MILLET:  Your Honor, I am going to move to 

20    strike that.  That is stating what Dr. Breiman may or may 

21    not know and that is beyond this witness' ken.  

22               MR. SOLOMON:  The witness is testifying as to 

23    what he understands.  This witness is surely in a position 

24    to know what someone who communicated with the Bureau about 

25    the PES was.  
                                                              626

 1               MR. MILLET:  He is not in a position to know what 

 2    Dr. Breiman knows.  

 3               THE COURT:  The factual statements I will take, 

 4    he had no contact, he was not a member of the Bureau, et 

 5    cetera.  The opinion part I will strike.  

 6               MR. MILLET:  Thank you.  

 7    BY MR. SOLOMON:

 8         Q.    You testified that the conclusion by the Bureau 

 9    that adjustment was technically feasible and that the Bureau 

10    would seek funding was published.

11               I would like you to take a look at PX 148, that 

12    is in tab 12, the document that we marked and admitted 

13    before, and I call your attention to the first paragraph.  

14    It says, "In this paper we discuss the issue of whether 

15    there exists a rigorous and professionally sound body of 

16    statistical theory, methods and operations for correcting 

17    the 1990 census enumeration so as to produce census figures 

18    with reduced differential undercount.  We show that such 

19    methods exist and that corrections to the census are 

20    technically feasible." 

21               I then am going to skip a couple of sentence and 

22    it says, "As a consequence of this judgment, the Census 

23    Bureau has decided to seek funding for a large-scale 

24    coverage measurement program for the 1990 census.  If this 

25    program is funded and successfully completed, it will be 
                                                              627

 1    used to correct the 1990 enumeration." 

 2               Is that one of the places where the Bureau 

 3    publicly disclosed its decision in the spring of 1987?

 4         A.    Yes, sir.

 5         Q.    Thereafter, did the Census Bureau issue a press 

 6    release disclosing the decision that had been made?

 7         A.    Director Keane planned a press conference to 

 8    announce this decision.  That was the end of May or early 

 9    June of 1987.  

10               I'm told that the Commerce Department asked him 

11    to cancel that press conference --  

12               MR. MILLET:  Objection; that is hearsay.  

13               THE COURT:  Sustained.

14         A.    In fact, I know that the press conference never 

15    occurred.

16         Q.    Fair enough.

17               You know that the press conference never 

18    occurred?

19         A.    Yes, sir.

20         Q.    How do you know that?  

21               THE COURT:  He wasn't asked.  

22               MR. SOLOMON:  He wasn't asked.  Fair enough.

23         Q.    Did you have any communications with anyone at 

24    the Bureau concerning why the press conference did not 

25    occur?  
                                                              628

 1               MR. MILLET:  It calls for hearsay, your Honor.  

 2               MR. SOLOMON:  Your Honor, we are talking about 

 3    the statements of a party opponent.  It is not hearsay.  

 4               THE COURT:  Party --  

 5               MR. SOLOMON:  Opponent.  

 6               THE COURT:  Overruled.

 7               You may answer.

 8         A.    Did I discuss the cancelling of the press 

 9    conference?

10         Q.    Yes.

11         A.    Yes.

12         Q.    With whom?

13         A.    With members of the board, with members of my 

14    staff, with other colleagues and friends around the Census 

15    Bureau.

16         Q.    What is your understanding of why the press 

17    conference was cancelled?

18         A.    My understanding is that the Commerce Department 

19    was reconsidering Director Keane's decision and it did not 

20    wish to announce a decision until it had completed its 

21    reconsideration.  

22               MR. MILLET:  Your Honor, objection and motion to 

23    strike, and if I may make a brief statement.  I know this 

24    was a subject of a motion in limine which we filed and I 

25    just want to make our objection clear on the record once and 
                                                              629

 1    I will accept your Honor's ruling.

 2               It is defendants' position that the events that 

 3    the witness is now testifying to, which I presume are 

 4    leading up to the department's initial decision not to 

 5    conduct an adjustment, are vitiated by the stipulation and 

 6    order in this case and are not relevant to the issues before 

 7    the court.  

 8               THE COURT:  I understand.

 9               The objection is overruled.  

10               MR. MILLET:  Can we make that a standing 

11    objection for the continuation of the trial?  

12               THE COURT:  Yes, you have a continuing objection.  

13    BY MR. SOLOMON:

14         Q.    Do you have the question, Dr. Wolter?  

15               THE COURT:  How could he?  

16               MR. SOLOMON:  Impossible.

17         A.    I have the objection, though.  

18               MR. SOLOMON:  Could we ask you, sir, to reread 

19    it.

20               (Question read)

21    BY MR. SOLOMON:

22         Q.    In the summer of 1987, were you told by someone 

23    in the Census Bureau that the Department of Commerce had 

24    decided to overrule the decision of the Bureau?

25         A.    Yes, sir, I was.
                                                              630

 1         Q.    Were you told at that time not to disclose that 

 2    fact to anyone?

 3         A.    Yes, sir.

 4         Q.    Did you thereafter attend professional meetings?

 5         A.    Yes, sir.

 6         Q.    Was the fact that you had earlier published the 

 7    fact that the Bureau had made a decision that you then were 

 8    instructed not to disclose the fact that the Census Bureau 

 9    had been overruled by the Department of Commerce and you 

10    thereafter attended professional meetings embarrass you?  

11               THE COURT:  I'm lost.  

12               MR. MILLET:  It's leading.  

13               THE COURT:  I don't have that question at all.  

14               MR. SOLOMON:  Fair enough.  

15               THE COURT:  Too long.

16         Q.    You attended professional meetings after you were 

17    told not to disclose the Commerce Department's decision, 

18    correct?

19         A.    Yes, sir.

20         Q.    Was the subject of the potential adjustment of 

21    the 1990 census via PES one of the subjects that was 

22    discussed at these conferences?

23         A.    Sure.  We presented this paper at the conference.  

24               THE COURT:  What paper? 

25               THE WITNESS:  148.  
                                                              631

 1               THE COURT:  Where are we in time here? 

 2               THE WITNESS:  This would be August of 1987.

 3               Just to give some sense of the timing, the 86 Los 

 4    Angeles work actually occurred in the summer of 86.  The 

 5    work was completed by the winter of 87.  Then the quality 

 6    checks were completed by May of 87, and then by August of 87 

 7    were presenting the decision and the results of the test at 

 8    the American Statistical Association meetings.

 9         Q.    What was your reaction to being at a meeting that 

10    presented that paper under instructions that the Commerce 

11    Department had overruled the decision, but that you could 

12    not disclose that?  

13               MR. MILLET:  This is a different kettle of 

14    relevance, your Honor, than my prior objection.  This is 

15    personal feelings are not relevant.  

16               THE COURT:  Overruled.

17               You may answer.

18         A.    Well, it was, it was uncomfortable at the least.  

19    Embarrassing?  Yes.  

20               THE COURT:  Why?  Why were you embarrassed if 

21    nobody knew that you knew? 

22               THE WITNESS:  Because friends, people that I had 

23    worked with for years are coming up to me and asking me, 

24    "What's the decision?"  And I'm saying on the one hand, 

25    well, here's the decision, this is what Director Keane 
                                                              632

 1    decided, and at the same time I know, I've been told by the 

 2    deputy director of the Bureau that the Commerce Department 

 3    has decided otherwise and he is telling me not to say 

 4    anything to anyone --  

 5               THE COURT:  Would uncomfortable be a better word 

 6    than embarrassed? 

 7               THE WITNESS:  Sure, uncomfortable.  

 8               THE COURT:  All right.  

 9               MR. SOLOMON:  I felt I was being dishonest with 

10    people I admired and respected.  

11    BY MR. SOLOMON:

12         Q.    In any of the discussions that you had with 

13    anyone at the Commerce Department or anyone at the Bureau of 

14    the Census concerning why the Commerce Department had 

15    overruled the Bureau's determination, did anyone offer you 

16    any reason based on scientific analysis or any of the data 

17    that you had generated?

18         A.    No.

19         Q.    You mentioned briefly yesterday that as a result 

20    of the stipulation the PES was put back on track.

21               You do have knowledge of how the PES was, in 

22    fact, carried out, correct?

23         A.    Yes, sir.

24         Q.    You talked earlier about the design of the PES 

25    including trying to maximize the independents of the PES 
                                                              633

 1    from the census.

 2               Do you recall that generally?

 3         A.    Yes.

 4         Q.    Was that independence achieved in the 1990 PES?

 5         A.    Yes.

 6         Q.    How many housing units were interviewed as part 

 7    of the 1990 PES?

 8         A.    170,000, give or take.

 9         Q.    And approximately how many people were covered by 

10    that 170,000 housing units?

11         A.    Something just under 400,000, I believe.

12         Q.    You described generally how the sample size was 

13    going to be selected, how the sample was going to be 

14    selected as part of the PES.

15               Was that sample design, in fact, utilized?

16         A.    Yes, sir.

17         Q.    In your opinion, is the 170,000 large --  is 

18    170,000 sample survey a large survey?

19         A.    It's a very large survey.  It's one of the most 

20    massive surveys ever conducted at the Bureau.

21         Q.    You described in general innovations that were 

22    made in connection with the matching and how the 1990 

23    designed for matching as part of the PES.

24               Was that design carried out?

25         A.    Yes, sir.
                                                              634

 1         Q.    You also talked about specially trained teams and 

 2    followup efforts as part of the design of the PES.

 3               Was that, in fact, carried out?

 4         A.    Yes, sir.

 5         Q.    You talked about the creation of post-strata as 

 6    part of the design to reduce heterogeneity bias.  Was 

 7    post-stratification carried out?

 8         A.    Yes, it was.  

 9               MR. MILLET:  A lot of leading questions, your 

10    Honor.  

11               THE COURT:  Overruled.

12         Q.    Look, please, at tabs 24 and 25 just for a 

13    moment.  Naturally, enough, they are in reverse 

14    chronological order.  

15               Let me start with 25.  For the record that is PX 

16    660.  It is memorandum from Kirk Wolter to the undercount 

17    steering committee, dated May 17, 1988.

18               Can you identify that document to the court?

19         A.    Surely.  It's another document that I wrote.

20         Q.    What is it?

21         A.    Well, it, among other things, it sets forth our 

22    first draft of the post-stratification scheme that might be 

23    utilized for the 1990 PES.

24               Again, remember, now, we are talking about 

25    chopping the country up into pieces so that people tend to 
                                                              635

 1    be alike within a piece, and this is our first, first draft 

 2    of that.

 3         Q.    Was a revision made to that and was a preliminary 

 4    stratificaion scheme arrived at?

 5         A.    Yes.

 6         Q.    Would you look, please, at the prior tab, PX 566 

 7    for identification, entitled, "Preliminary stratificaion 

 8    schemes for the 1990 census coverage by measurement 

 9    programs." 

10               Can you identify that for the court?

11         A.    Sure.  It's the same thing, roughly.

12         Q.    In the first of the documents, PX 660, you 

13    state, "I plan to seek comments from external experts as of 

14    July 1." 

15               I take that to be 1988.

16               Was that, in fact, done?

17         A.    Yes, it was.

18         Q.    Do you recall any reactions or responses or 

19    comments by external experts with respect to the 

20    stratificaion scheme?

21         A.    There were a few miscellaneous expressions of 

22    support.

23               I talked with some experts one-on-one who gave me 

24    some feedback, generally support.  

25               I don't remember any, any contrary advice, I 
                                                              636

 1    don't remember any criticism, and I don't remember receiving 

 2    anything in the way of major written criticism or even major 

 3    written support.

 4         Q.    Are PX 660 and 566 precursors of what became the 

 5    actual stratificaion into 1,392 post-strata as used as part 

 6    of the 1990 PES?

 7         A.    Yes, sir.  

 8               MR. SOLOMON:  Plaintiffs would offer 660 and 566.  

 9               MR. MILLET:  I think 566 may already be in.

10               I have no objection to 566.

11               660 I will note, your Honor, although the witness 

12    identified it as something he has written, on its face it 

13    appears to have a number of documents included that are 

14    written by various people and on that basis --  well, I will 

15    just point that out for the court, your Honor, I won't 

16    object to it.  

17               THE COURT:  All right.  660 is admitted.  I grasp 

18    Mr. Millet's point.  

19               (Plaintiff's Exhibit 660 marked  or 

20    identification was received in evidence.)

21    BY MR. SOLOMON:

22         Q.    Look please at the next two tabs, tabs 26 and 27.  

23    They are for the record PX 94, a December 13, 1990 

24    memorandum from Howard Hogan to the undercount steering 

25    committee, and the next tab, tab 27 is PX 162, which for the 
                                                              637

 1    record is a December 21, 1990 note from Dr. Wolter to Dr. 

 2    Hogan.

 3               Can you identify those documents?

 4         A.    Yes.

 5               The first is a memo by Dr. Hogan.  We received it 

 6    as members of the special advisory panel.  It describes some 

 7    technical details of the process of removing sampling 

 8    variability from PES data.

 9               The very last tab is a little note that I wrote 

10    to Howard Hogan providing my reaction to the December 13, 

11    1990 memorandum.

12         Q.    You testified that the removal of the sampling 

13    variability that you are talking about is what we have been 

14    talking about as the smoothing, is that right?

15         A.    Yes, sir.

16         Q.    Did I recall correctly that you testified that 

17    smoothing was, in fact, tested as part of the 1986 TARO?

18         A.    Yes, and 88.

19         Q.    And the 88 dress rehearsals?

20         A.    Yes.

21         Q.    Thank you.

22               I refer in your note, Plaintiff's Exhibit 162, 

23    where it says, "Looks like the same post-strata and carrier 

24    variables and have been under discussion since 1988 or so, 

25    with the exception of new post-strata for the Asian 
                                                              638

 1    community." 

 2               The "carrier variables" you are referring to 

 3    there, is that part of the smoothing process?

 4         A.    Yes, it is.  

 5               MR. SOLOMON:  We would offer PX 94 and 162.  

 6               MR. MILLET:  No objection, your Honor. 

 7               If I may, I have been told I may be wrong in 

 8    saying that 566 was already in, in which case we don't 

 9    oppose it.  

10               THE COURT:  566 is in if it isn't already, and 94 

11    and 162 are admitted.  

12               (Plaintiff's Exhibits 94, 162 and 566, 

13    respectively, marked for identification were received in 

14    evidence.)

15    BY MR. SOLOMON:

16         Q.    When you were describing the 1986 tests, you 

17    talked about taking adjustment factors and adjusting down to 

18    subunits or subareas.

19               Do you know whether that was done in the 1990 

20    PES?

21         A.    It was, indeed, done.

22         Q.    As a member of the special advisory panel, did 

23    you conclude that the process by which the data was brought 

24    down to lower levels of geography was done appropriately?

25         A.    Yes, it was.
                                                              639

 1         Q.    Can you tell us the basis for that opinion?

 2         A.    Well, by the word "appropriately" means to me 

 3    that it's done in such a way that the resulting small area 

 4    estimates of total population ameliorate the differential 

 5    undercount present in the original enumeration and in my 

 6    judgment these data did that.

 7         Q.    Was there theoretical work done in connection 

 8    with the analysis that the Bureau did supporting the 

 9    proposition that this trickling down or this adjustment 

10    downward would improve the data?

11         A.    Yes, there was some theoretical work, there was 

12    some empirical work.

13         Q.    Tell the judge about the theoretical and 

14    empirical work.

15         A.    One of my colleagues, John Tukey, initiated some 

16    theoretical work.  Tukey is not a member of the Census 

17    Bureau, but nonetheless initiated some theoretical work in 

18    the early 1980s.

19               Then in the 87-88 time frame I initiated some 

20    theoretical work in conjunction with a member of my staff, 

21    who we haven't yet discussed today, and that is Beverly 

22    Causey.  And Bev and I carried forth some theoretical work 

23    as well as empirical work.

24         Q.    Did Isaki do any empirical work?

25         A.    Isaki is the person and the staff head who was 
                                                              640

 1    assigned the primary responsibility for empirical work on 

 2    the trickling down process throughout the entire decade of 

 3    the 1980s.  He was responsible for formulating the right 

 4    questions, he was responsible for determining approaches to 

 5    those questions, he was responsible for answers, he was 

 6    responsible for designing and implementing the final 

 7    trickling down process implemented in the 1990 

 8    post-enumeration program.

 9         Q.    Do you know whether in any of Dr. Isaki's work 

10    he, in fact, counted up the number of states as discrete 

11    political units and without regard to size and test whether 

12    the trickling down thesis applied in that context as well?  

13               MR. MILLET:  Do we have a time frame for this?  

14               MR. SOLOMON:  Yes.  Ever.  

15               MR. MILLET:  Ever?  

16               MR. SOLOMON:  Ever.  

17               MR. MILLET:  In the history of mankind?  

18               THE COURT:  Ever.  

19               MR. SOLOMON:  I don't believe the question is 

20    objectionable.  You keep asking me.

21         A.    Well, Isaki carried out a very substantial number 

22    of empirical studies of alternative methods of correcting 

23    small area census data and the quality of those corrections 

24    and he very carefully evaluated the impact of the correction 

25    process on the quality of state level census counts, county 
                                                              641

 1    level census counts, enumeration district level census 

 2    counts, and I believe later in the decade even looked at 

 3    some block level census counts.

 4         Q.    What conclusions did he reach?

 5         A.    It would be very difficult for me to summarize 

 6    all of them, but he generally identified methods of 

 7    trickling down the measured undercount to the block level or 

 8    to the local level that function well, well in the sense 

 9    that they ameliorate the differential undercount in the raw 

10    counts.

11               He also implemented multiple measures of better, 

12    of good; multiple, to use your word, loose functions to try 

13    and understand what the properties of small area corrections 

14    are.  

15               Do small corrections improved under all 

16    definitions of better or under half of the definitions and 

17    not the other half or only under one definition, but make 

18    things worse under 37 other definitions of better?  

19               And generally speaking he found that the range of 

20    trickling down methods, correction methods that he was 

21    looking at broadly improved under all methods or under all 

22    loose functions, that is, under all definitions of better. 

23               I'm sure he found other things, too, and if 

24    pressed to the wall I can probably reconstruct some of them, 

25    but these are some of his principal findings.
                                                              642

 1         Q.    You have testified to your conclusion concerning 

 2    whether the PES was properly implemented as parts of the 

 3    1990 census.

 4               I would like for you to explain to the court the 

 5    basis for the opinion that you have expressed about that.

 6         A.    Would you repeat the question?

 7         Q.    It is easier just to go back.

 8               Did you reach a conclusion concerning whether the 

 9    PES was properly implemented?

10         A.    Properly implemented?

11         Q.    Yes, sir.

12         A.    Yes.  As a panel member we reviewed the 

13    implementation regularly and I concluded that.

14         Q.    Will you describe for the court the basis for 

15    your conclusion that the PES was properly implemented?

16         A.    I was a member of the panel that this court 

17    established.  As a member of the panel I was the beneficiary 

18    of Census Bureau materials on the conduct of the original 

19    enumeration and the conduct of PES.

20               In addition, the panel met with the Census Bureau 

21    staff regularly.  We received status reports on the conduct 

22    of the original enumeration and on the conduct of the PES.

23               In many cases the progress reports were presented 

24    to us by people that I had hired and trained or by people 

25    who had worked for me or by others who I had confidence in 
                                                              643

 1    elsewhere in the Bureau.

 2               Throughout this entire period I was led to 

 3    believe by the materials presented to me as well as by the 

 4    people I had confidence in within the Bureau that everything 

 5    was going very well, PES was going well, the listing went 

 6    well, the interviewing went well QC went well.  It went well 

 7    all the way through every step of the process.  There 

 8    wasn't, there wasn't any point in the process that I recall 

 9    any major concern, any major flaw.  PES was designed to go 

10    well and evidently it did.

11         Q.    The QC that you mentioned, that was the quality 

12    control checks?

13         A.    Excuse me.  Those were the in-production quality 

14    checks as opposed to the post-production quality checks.

15         Q.    Were you interested in the question whether the 

16    PES results were consistent with your prior expectations?

17         A.    I examined that and my panel colleagues examined 

18    that issue, yes.

19         Q.    I would like you to explain to the court why it 

20    was relevant to you to know whether the PES results were 

21    consistent with your expectations?

22         A.    Well, as I described it yesterday, there was a 

23    very strong and regrettably persistent pattern of 

24    differential undercounting in the United States in spite of 

25    the Census Bureau's very good efforts which are, indeed, 
                                                              644

 1    very good, and this has gone on for decades.

 2               I didn't expect any major departure from that 

 3    historical pattern in 1990, and I wanted to learn and see 

 4    whether the PES was consistent with this very strong pattern 

 5    of undercounting.

 6               Had the PES displayed a very different pattern of 

 7    undercounting than expectation, then that would have 

 8    required real digging and real explanation.  

 9               On the other hand, to the extent that the PES was 

10    confirming this enormously persistent pattern, then one 

11    begins to gain confidence, one, I should say, begins to gain 

12    even further confidence that the PES implementation went 

13    well, the design worked as designed, the design resisted 

14    flaw, et cetera.

15         Q.    What were some of the results of the PES that you 

16    looked at that in your judgment confirmed your expectation?

17         A.    Well, very quickly, minorities tend to be 

18    undercounted at a higher rate than nonminorities, blacks in 

19    particular, nonblack Hispanics in particular and other 

20    communities, too, tend to be undercounted at a higher rate 

21    than nonminorities.

22               Men, regrettably, tend to be undercount at a 

23    higher rate than women.  The young and mobile, regrettably, 

24    tend to be undercounted at a higher rate than more stable 

25    age groups in our society.  Central cities tend to be 
                                                              645

 1    miscounted to a greater extent than suburban areas with well 

 2    defined housing units, clearly marked addresses and so on.

 3               Areas where there was greater cooperation with 

 4    the original enumeration tend to have lower undercounts.  

 5    Areas that had relatively less cooperation with the original 

 6    enumeration tend to have higher undercounts.

 7               We looked at that by looking at what's called the 

 8    mail-back rate.  Mail-back refers to the number of people 

 9    who send back their census form in the original enumeration.

10               Difficult areas are populated by people that 

11    don't mail their form back.  Undercount tends to be high 

12    there historically, and it certainly was in 1990 as well.

13               The poor, the disadvantaged, the illiterate, 

14    these are the groups that tend to be undercounted, and 

15    that's a very strong, very powerful, very persistent decade 

16    after decade after decade pattern and the pattern continued 

17    in 1990.

18         Q.    Generally speak, did PES confirm those 

19    expectations?

20         A.    Yes, sir, it did.

21         Q.    You mentioned the design of the PES.

22               Did anything ever come to your attention that 

23    suggested that the design was flawed in some significant 

24    respect?

25         A.    No.  This, this was a design that resulted from 
                                                              646

 1    literally decades of refinement and intensive refinement 

 2    this decade and extensive testing this decade and it proves 

 3    to be very resistant to difficulty, to flaw.

 4         Q.    Were there are specific post-production quality 

 5    control checks of the quality of the PES?

 6         A.    Yes, there were of the kind I described earlier 

 7    in 86 and 88.

 8         Q.    Can you describe for the court generally what 

 9    those studies entailed? 

10               There has been some testimony about this already.  

11    I think you can describe generally what those studies 

12    entailed.

13         A.    All right.  There were studies of missing data, 

14    there were studies of matching, there was studies of 

15    misreporting of census address and all of these other 

16    components of error that I described earlier.

17               There were on the order of 20 or more quality 

18    checks of all these components of error.

19               The Census Bureau called these quality checks P 

20    studies, and in their literature, in their memoranda you 

21    will see them called study P1, P2, P20, and there were --  

22    we haven't talked about the demographic analysis estimates 

23    of the undercount, but there were further studies of 

24    demographic analysis and they came to be known as D1, D2, et 

25    cetera.
                                                              647

 1         Q.    What conclusion, if any, did you reach concerning 

 2    the quality of the PES from your review of the P studies and 

 3    the studies on demographic analysis?

 4         A.    I concluded that the margin of error in the PES 

 5    was extremely small.

 6               Now, I've said PES went very well, I've said it 

 7    was bulletproof.  

 8               I'm not going to sit here and tell you it's 

 9    perfect, I'm not going to sit here and tell you it doesn't 

10    have any nonsampling error in it.  

11               But I am going to tell you it was extremely good.  

12    It's one of the best surveys that I have every witnessed in 

13    my professional career, and I think the Census Bureau 

14    deserves enormous credit for that.  And all of these 

15    studies, I think, demonstrate that very, very low margin of 

16    error.

17               The margin of error in the PES is much smaller 

18    than the margin of error in the original enumeration, and 

19    that's what allows the corrected counts to ameliorate the 

20    differential undercount.

21         Q.    I don't want to go in the P studies in any more 

22    detail, but I would like to call your attention to a few 

23    statements in the Secretary's report and ask you what your 

24    opinion of them is based on your review of the P studies.

25               The excerpts are in tab 3 of the book that is in 
                                                              648

 1    front of you, and the first one is on page 2-16.

 2               All the way at the bottom of the page, the last 

 3    sentence says, "The imputation scheme used by the PES is 

 4    based on a series of assumptions that are mostly guesswork." 

 5               Would you give the court your reaction to that 

 6    and identify for the court whether there are P studies that 

 7    form the basis of your opinion?

 8         A.    Well, there are P studies that form the basis for 

 9    my opinion, yes, and the statement is wrong.

10         Q.    Why?

11         A.    Well, first of all, the imputation that was used 

12    in the PES is the same as or better than the kind of 

13    imputation used in any other survey or census.  It's based 

14    on dozens of papers and statistical journals, it's based on 

15    an enormous body of scientific evidence, and to call it 

16    guesswork is just off the mark.  To call it guesswork is 

17    guesswork.  

18               And to say that it's based on a series of 

19    assumptions --  well, that's true.   As I said yesterday, we 

20    don't know the missing data, if we did we would use it, so 

21    instead we have to estimate the missing data.

22               Fortunately, in the PES there was very little 

23    missing data in the first place that we had to estimate.

24               There still remains a question, though, as to 

25    whether we have estimated the data correctly or not, and to 
                                                              649

 1    test that the Bureau mounted quality checks and it estimated 

 2    the missing data in a number of different ways under a 

 3    number of different reasonable assumptions, and what it 

 4    found was it didn't matter.  

 5               Under a range of reasonable assumptions, given 

 6    the low level of missing necessary, it just plain didn't 

 7    matter, all the alternatives led to the same conclusions 

 8    about the size of the undercount.

 9         Q.    Turn to the next page in the Secretary's report 

10    under his discussion of matching error.  In his last 

11    sentence, you can read as much as you need to put it in 

12    context, it says, "His evidence, though anecdotal, is 

13    suggestive of the fact that the variance due to matching 

14    error is conservatively estimated in the total error model." 

15               Would you comment on that, please?

16         A.    Well, the Bureau mounted very significant, very, 

17    very significant in-production quality checks on the 

18    matching operation, and then even going beyond that, the 

19    Bureau implemented significant quality post-production 

20    quality checks of the quality of the matching.  Several of 

21    these P studies dealt in matching error, and these studies 

22    demonstrated that matching error was controlled to extremely 

23    low levels that in no way compromises the utility of PES for 

24    reducing the differential undercount.  And I don't know why 

25    we are relying on anecdotal evidence at this point in light 
                                                              650

 1    of that, in light of that massive amount of good scientific 

 2    work by the Bureau.  That's absurd.

 3               I know of no scientific study that demonstrates 

 4    that matching error has been conservatively estimated.

 5         Q.    Let me ask you to turn to one more page in the 

 6    Secretary's decision, it's on page 2-45.  The statement 

 7    appears on page 2.

 8               The Secretary says, "Furthermore, substantial 

 9    evidence casts doubt on the hemogeneity assumption 

10    underlying the entire synthetic adjustment methodology." 

11               The synthetic adjustment methodology is that 

12    trickling down to the small areas that you testified to 

13    before?

14         A.    Yes, sir.

15         Q.    Are you aware of any evidence that casts doubt on 

16    the homogeneity assumption underlying the synthetic 

17    adjustment methodology?

18         A.    Well, let me put it this way:

19               Common sense casts doubt on it.

20               I mean, we know for reasons I said earlier that 

21    there is always some inevitable limited heterogeneity.  We 

22    know that.  And you pointed me to passages earlier that 

23    tells us that that is inevitable.

24               Nonetheless, the Bureau devised what, in my 

25    judgment, is a very good post-stratification scheme.  It 
                                                              651

 1    divides the country up into cells wherein people generally 

 2    are alike within cell.  Maybe there is some residual 

 3    heterogeneity within cells, but notwithstanding that, 

 4    notwithstanding that, the PES is still better than is the 

 5    original original enumeration.

 6               One cannot let, one cannot in this case let 

 7    somehow the best become the enemy of the better.

 8         Q.    What do you mean by that?

 9         A.    That is to say, if there were some hypothetical 

10    methods that could eliminate homogeneity, it would be all 

11    the better, and to say that failure to achieve that best 

12    should get in the way of a PES that is better than the 

13    original enumeration is nuts.

14         Q.    You told us before about the work on the total 

15    error model summing up all of the errors in the PES.

16               Was that analysis also undertaken after the 1990 

17    PES?

18         A.    Yes, sir.

19         Q.    Would you turn to tab 22 in your binder and there 

20    you will find a portion of PX 540, which has already been 

21    admitted.

22               Can you tell the court what -- 

23               THE WITNESS:  Your Honor, may I take my jacket 
      off again?  
24               THE COURT:  Sure.  A five minute break would be 
      appreciate at this point.
25               (Continued on the next page.)
                                                              652

 1               (In open court) 

 2    KIRK WOLTER, resumed

 3    DIRECT EXAMINATION CONTINUED 

 4    BY MR. SOLOMON:
                     

 5         Q.    We were just turning to tab 22.

 6               THE COURT:  Tab 22.

 7         Q.    Right, which is PX 520.  What is that document, 

 8    sir?

 9         A.    It's a memorandum, an internal Census Bureau 

10    memorandum from Dave Baitman to John Thompson.  It describes 

11    one of these P-Studies known as P 16, and, it's one of the 

12    principal documents that sums up all the quality checks into 

13    an overall integrated view of the error in the original 

14    enumeration and in the PES.

15         Q.    Could you just leaf through or walk through the 

16    document for the Judge, just give him a sense of what the 

17    analysis in this document entails?

18         A.    Okay.  Brief walk through?

19         Q.    Please.

20               THE COURT:  Jog.

21         A.    Jog.

22               Okay.  Well, let me simply call your Honor's 

23    attention to page 7.  There's a section 4 here that 

24    discusses components of PES error.  And for the next several 

25    pages, you will see sections that are headed by the names of 
                                                              653

 1    the error components that I described earlier.  This model 

 2    error is the thing that I called correlation bias earlier.

 3               Moving on through pages 8 and 9, you'll see 

 4    matching error.  

 5               On 10 you'll see error in reported Census Day 

 6    Address.

 7               On 12, you'll see fabricated interviews 

 8    discussed.

 9               On 13 and 14 you'll see discussions of errors 

10    that may or may not have been committed in the measurement 

11    of those people who were inappropriately counted in the 

12    original enumeration.

13               Missing data is discussed on 14 and 15.  And so 

14    on it goes.  And as we go, you'll see a lot of texts, a lot 

15    of equations, and so on.  But the general thrust of all this 

16    is that we've conducted quality checks on all of the errors 

17    that really matter in the PES.

18               Then the paper tries to sum up the quality checks 

19    to an overall integrated view of the error, and it does this 

20    within each of 13 pieces of the country.  And I would direct 

21    your attention to the bottom of page 18, where these 13 

22    pieces of the country are set forth.

23               No. 1, for example, is the northeastern part of 

24    the country, central cities in the northeastern part of the 

25    country, and the minority population within central cities 
                                                              654

 1    within the northeastern part of the population.  That's one 

 2    of these 13 pieces.

 3               The Census Bureau conducted all of these quality 

 4    checks within each of the 13 pieces, and then this document 

 5    proceeds to sum up and develop an overall integrated view of 

 6    the total error within each of the 13 pieces.

 7               That overall integrated view of error then is 

 8    presented in numerical tables at the end of this document -- 

 9    in fact, by my reckoning, there's table 1 through table 20, 

10    and tens if not hundreds of numbers on each page, and these 

11    convey the results of the Census Bureau's careful analysis 

12    of total error.

13         Q.    Is there a table that then totals up the errors 

14    from the various evaluation poststrata as identified to 13 

15    areas of the country in this document?

16         A.    Yes, sir.

17         Q.    Would you explain that to the Judge.

18         A.    Both tables 17 and table 19 do that.  Table 17 

19    does it before the Bureau stripped out the sampling error.  

20    Table 19 did it, again, after -- excuse me.

21               Let me revise what I just said.  Table 17 and 

22    table 15 sum it up in a convenient form.

23               Let's look at 17.  17 sums it up after the Bureau 

24    has stripped out this sampling variability, or as much as it 

25    could, by way of the so-called smoothing operation.  
                                                              655

 1               Shall I just tell you about three or four numbers 

 2    in here?

 3         Q.    Please.  See if you can give the Judge a sense of 

 4    what's being presented and what conclusion you can draw from 

 5    it.

 6         A.    Okay.  Well, although the issue before us is 

 7    differential undercount, let me keep this simple and just 

 8    talk about the national undercount.  Okay?

 9         Q.    Sure.

10         A.    I think I'll be able to make my points there, 

11    even though the real issue is differential undercount, 

12    although maybe I can do that too.

13               If you look at the bottom of table 17, you'll see 

14    a row called "National."  That's the whole of the U.S.  And 

15    there is a number there, 2.1.  That's the estimated national 

16    undercount.  2.1 percent.

17         Q.    And that U with a hat on top, what's that?

18         A.    U just stands for undercount.  The hat is just 

19    notations that statisticians use. 

20         Q.    Okay.

21         A.    The next number is .4.  Now, that is one of the 

22    key results from the summing up of errors.  We've summed up 

23    all of the components of errors now, and we have summarized 

24    them in two principal ways, and this is very common for 

25    statisticians to do.
                                                              656

 1               One is called the bias and the other is called 

 2    variability, or a measure of that is what a statistician 

 3    would sometimes call standard error or standard deviation.

 4               First of all, the bias.  From these quality 

 5    checks, the Bureau is estimating -- these post-production, I 

 6    should say, quality checks -- the Bureau is telling us that 

 7    their best estimate of systematic error or bias in the PES 

 8    data is four-tenths of 1 percent.  Okay?

 9         Q.    All right.

10         A.    Now, they are telling us that the estimated 

11    undercount of over 2 percent may be a little high, maybe 

12    around four-tenths percent too high.

13               The next two numbers, the .1 and the .2 simply 

14    refer to measures of this second property of error, which I 

15    call variability, which I said is measured by standard 

16    deviation or standard error.  And the Bureau's best estimate 

17    of that ultimately is the .24 that appears in the column 

18    headed "Total Standard Deviation."  Okay?

19         Q.    Yes.

20         A.    All right.  And then finally, in the last column, 

21    you'll see two numbers, 1.2 and 2.2, with parentheses around 

22    them.  That is intended to represent an interval, the 

23    interval of all undercounts between 1.2 percent and 2.2 

24    percent.

25               Statisticians sometimes call such intervals 
                                                              657

 1    confidence intervals, where statisticians call the 2.1 

 2    percent that appears over on the left the point estimate, 

 3    the best estimate of the undercount.  

 4               Now, the confidence interval is simply intended 

 5    to represent an interval within which statisticians are 

 6    virtually certain that the truth lies; that is to say, if 

 7    this entire PES census enterprise were repeated thousands 

 8    and thousands and thousands of times, damn near all -- 

 9    excuse me -- almost all of these intervals would contain the 

10    truth.  So it's often useful to have not only the best 

11    estimate, the point estimate, namely, 2.1 percent, but also 

12    an interval, and it's very common in Census Bureau work as 

13    well as other statistical work to produce intervals, and 

14    here we have one.

15               Now, if you look on the next page, I noticed a 

16    way that I can convey to you some brief information 

17    regarding the differential undercount.  This is table 18, 

18    and you'll see here two rows.  One is entitled "minority" 

19    and the other is entitled "nonminority."

20               And the next column tells us what the estimated 

21    undercount is.  It's 5.6 percent for minorities, it's 1.1 

22    percent for nonminorities.  Here it is in black and white.  

23    This is the differential undercount.  This is the reason why 

24    we are all here.  This is the persistent strong pattern of 

25    differential undercounting that has occurred since 1940.
                                                              658

 1         Q.    What is -- 

 2               MR. MILLET:  Move to strike as to why we are all 

 3    here, your Honor.

 4               THE COURT:  I'll strike it.

 5         Q.    What is the chart telling us about the 

 6    differential?

 7         A.    Well, it's telling us the differential is 5.6 

 8    minus 1.1 percent.  It's 4 1/2 percentage points difference.

 9               Then the Bureau tells us what their best evidence 

10    is regarding the two properties of PES error.  The bias 

11    property is given in the column headed B, and the 

12    variability property of error is presented in the column 

13    headed "Total Standard Deviation."  And you can see that the 

14    bias in minority undercount is six-tenths of a percent.  

15    It's much higher than that in the case of nonminorities.

16               In the standard deviations, for minorities are 

17    about .7 percent, and about .5 percent, that's half of one 

18    percent respectively.

19               And then you see these 95 percent intervals given 

20    over on the right-hand side.  For minorities, it's just 

21    under 4 percent to well over 6 percent.  And for 

22    nonminorities, it's minus 1 percent up to six-tenths of a 

23    percent.  Now, the minus 1 percent deserves some 

24    clarification.

25               Whenever you see a minus undercount in any of the 
                                                              659

 1    Census Bureau literature, that refers to an estimated 

 2    overcount.  Overcounts, again, being instances where too 

 3    many people were duplicated, triplicated, or were counted 

 4    and shouldn't have been.  Those were overcounts.

 5               So, these data convey the summing up of the 

 6    overall errors for minorities and nonminorities, and you see 

 7    in black and white the differential undercount.

 8         Q.    Thank you.

 9               Let's focus on the 5 percent confidence interval 

10    that you made reference to.  

11               Is 95 percent the only confidence interval that 

12    was used at the Bureau?

13         A.    No.  In fact, I didn't think it was used anymore 

14    at the Bureau.

15         Q.    What is your understanding about what confidence 

16    interval is used at the Bureau?  

17               MR. MILLET:  I'm going to object to this, your 

18    Honor.  This is a Bureau document using 95 percent 

19    confidence interval.  It speaks for itself.

20               THE COURT:  You may answer.  You can clarify what 

21    you see to be the confusion.

22               THE WITNESS:  Well, different confidence levels 

23    or different percentage levels here will be utilized by 

24    different people for different purposes, both within the 

25    Census Bureau and outside.
                                                              660

 1               During the past decade, when I was still at the 

 2    Census Bureau, the Census Bureau opened up this issue for 

 3    debate and discussion, and ultimately changed its standard 

 4    press releases from 95 percent to 90 percent.  And this 

 5    simply results in narrower intervals, and that, so far as I 

 6    know, has been the Census Bureau policy ever since.

 7         Q.    At whose suggestion was the rule of thumb changed 

 8    from 95 to 90?

 9         A.    Well, population and economic analysts within the 

10    Census Bureau and within the Commerce Department.  There is 

11    always a tension, shall we say, between substantive analysts 

12    of data and statisticians.  The substantive analysts want to 

13    at times draw inappropriate conclusions from data, and the 

14    statistician at times wants the substantive analysts to draw 

15    only appropriate conclusions from data.  

16               MR. MILLET:  Move to strike as nonresponsive and 

17    without foundation of the witness's knowledge as to what the 

18    current policy of the Bureau is concerning confidence 

19    intervals.

20               THE COURT:  Overruled.  

21    BY MR. SOLOMON:

22         Q.    Are rules of thumb or confidence intervals other 

23    than 95 and 90 used for important purposes in your opinion?

24               THE COURT:  Important purposes?

25         Q.    Important purposes.
                                                              661

 1         A.    Well, sure.  I mean, major decisions are made 

 2    within the Bureau every day, at different confidence 

 3    intervals than 90 or 95 percent.

 4               Actually, I can tell you in my own industry, and 

 5    now the marketing and media research industry.  The industry 

 6    tends to use what are sometimes called "one sigma confidence 

 7    intervals."  This would correspond to something like 67 

 8    percent intervals.  But a range of users and a range of 

 9    circumstances under different purposes will simply use 

10    different levels.  

11               MR. SOLOMON:  Thank you very much.  I have 

12    nothing further.

13               THE COURT:  Thank you.

14               I guess we should take lunch now.  Start cross 

15    afterwards.  

16               MR. MILLET:  That will be fine, your Honor.

17               THE COURT:  All right.  We'll resume at two 

18    o'clock.

19               (Luncheon recess) 

20               (Continued on next page) 

21    

22    

23    

24    

25    
                                                              662

 1                        AFTERNOON SESSION 

 2               (In open court)

 3               THE COURT:  Good afternoon.

 4    KIRK M. WOLTER, resumed
                             

 5               MR. MILLET:  May it please the Court.

 6               THE COURT:  Yes 

 7    CROSS-EXAMINATION

 8    BY MR. MILLET:

 9         Q.    Hello.

10         A.    Howdy.

11         Q.    Dr. Wolter, you referred to a P Study called P 20 

12    this morning.  Can you tell us what was the P 20 studied?  

13               MR. SOLOMON:  Objection.  No foundation.  

14               MR. MILLET:  He did refer to a P Study called 

15    P 20 this morning.  I'm asking him what P Study that was.

16         A.    I don't recall its title offhand.

17         Q.    Do you know if in fact there was a P 20?  

18               MR. SOLOMON:  I object.  There's no foundation to 

19    the question that the witness referred to P 20.  That's the 

20    form of my objection.

21               THE COURT:  Well, I don't really know.  But I'll 

22    permit him to go ahead.  It's cross-examination.

23    BY MR. MILLET:

24         Q.    My question --

25         A.    There were in the neighborhood of 20 P-Studies, 
                                                              663

 1    quality checks done after the PES.

 2         Q.    But you don't know whether it was one 

 3    specifically numbered P 20?

 4         A.    I don't recall at the moment, no.

 5         Q.    Is it your testimony that everything in the PES 

 6    worked?

 7               THE COURT:  Everything?

 8         Q.    Everything in the PES worked.

 9               THE COURT:  Worked?

10         A.    What do you mean by "worked"?

11         Q.    Operated according to plan.

12         A.    My testimony is that the PES-based correction 

13    ameliorates the differential undercount.

14         Q.    So your answer is yes?

15         A.    My answer is what I answered.

16         Q.    Can you answer my question yes or no, either it 

17    worked or didn't work?

18               THE COURT:  I'm not sure that worked/didn't work 

19    is a question that evince a yes no answer.  

20               MR. MILLET:  All right.  I'll withdraw the 

21    question.

22         Q.    Let me direct your attention to P Study P 16.  I 

23    believe that's at tab 22 of the binder that you were using 

24    with Mr. Solomon today.

25               THE COURT:  Plaintiffs' Exhibit 540.  
                                                              664

 1               MR. MILLET:  540 A 16, I believe it is, your 

 2    Honor.

 3         Q.    And I particularly would like to direct your 

 4    attention to table 17.

 5         A.    All right.  My tab 16 is empty.

 6         Q.    No tab 22?

 7         A.    Oh, I'm sorry.

 8         Q.    My tab 16 is empty too, Dr. Wolter.

 9         A.    Did I mishear you?  I'm sorry.

10         Q.    I'd like you to look the tab 22.

11         A.    Okay.

12         Q.    Exhibit 540 A 16, table 17.

13               Now, Mr. --

14               THE COURT:  And those are the tables you finished 

15    your direct examination with.

16         Q.    Right.

17         A.    Yes.

18         Q.    Those are the tables that you discussed with Mr. 

19    Solomon, correct?

20         A.    Correct, right.

21         Q.    I'd like to direct your attention to the same 

22    line as you were discussing with Mr. Solomon, and that's the 

23    last line which is entitled "National," and I want to the 

24    particularly direct your attention to the last column 

25    entitled, "95 percent Interval." 
                                                              665

 1         A.    Yes, sir.

 2         Q.    Does this indicate to you that it is virtually 

 3    certain that the true national undercount lies somewhere 

 4    between 1.23 percent and 2.2 zero percent?

 5         A.    Are you asking that in a formal or informal 

 6    sense?

 7         Q.    I'm asking you just as I asked it.  

 8               Does that indicate to you that -- strike that.

 9               That indicates to you, does it not, that the true 

10    national undercount lies somewhere between those two 

11    figures?

12         A.    Well, taking those two figures as numbers, the 

13    national undercount either lies in that interval or it 

14    doesn't.

15         Q.    Well, didn't you testify that 95 percent 

16    confidence interval indicates that it's virtually certain 

17    that the true national count lies somewhere between those 

18    figures in your direct examination this morning?

19         A.    The information I tried to convey this morning 

20    was that the 95 percent interval given here is virtually 

21    certain of covering the truth, and that if you repeat this 

22    entire enterprise over and over and over again, virtually 

23    all of the intervals so constructed will contain the truth, 

24    yes.

25         Q.    So the answer to my question is yes?
                                                              666

 1         A.    Your question was sufficiently imprecise that a 

 2    yes or no answer is insufficient.

 3         Q.    Well, take a look at the next table, table 18, 

 4    the nonminority line there.             

 5               Does that indicate to you that it's virtually 

 6    certain that the true count for nonminorities lies somewhere 

 7    between an overcount of 1.39 percent and an undercount of 

 8    point 62 percent?

 9         A.    That's -- that interval has the same meaning as 

10    the other interval does.

11         Q.    Thank you, Dr. Wolter.

12               Dr. Wolter, you described, I think over the last 

13    two days, really, the process of reaching what I think you 

14    term as a consensus during the decade of the 1980s on the 

15    procedures to be employed for the 1990 PES, is that correct?

16         A.    Yes.

17         Q.    And all of that consensus-building process was 

18    before any PES data from the 1990 post-enumeration survey 

19    were available, isn't that right?

20         A.    Yes, that's correct.

21         Q.    In fact, weren't there some descents from that 

22    consensus, Dr. Wolter?

23         A.    There were a range of views.

24         Q.    Didn't the census advisory committee on 

25    population statistics recommend, in the spring of 1987, that 
                                                              667

 1    adjustment could not be done by December 31st, 1990?

 2         A.    Their principal concern was the timing of the 

 3    process, yes.

 4         Q.    Am I correct in understanding your testimony from 

 5    yesterday that no reasonable statistician who had access to 

 6    the data you saw could disagree with your conclusion 

 7    concerning adjusting the 1990 Decennial Census, is that 

 8    correct?

 9         A.    I think I might have said that, yes.

10         Q.    So if the director of the Census Bureau were to 

11    advise the Secretary of Commerce that reasonable 

12    statisticians could differ on the question of adjusting 1990 

13    Decennial Census, that is incorrect in your view?

14         A.    I suppose she would.

15         Q.    Even if she looked at the same data you looked 

16    at, do you still believe she was incorrect if she advised 

17    the Secretary that?

18         A.    If she looked at the same data I did, yes.

19         Q.    Now, you testified yesterday that the Secretary 

20    was wrong to say that the tools used in the post-enumeration 

21    survey were on the cutting edge of statistical theory and 

22    survey methods, isn't that correct?

23         A.    Yes, sir.

24         Q.    And it's also your testimony, I take it, from 

25    yesterday that many of these tools and theories go back to 
                                                              668

 1    the 1940s or even earlier than that, is that accurate?

 2         A.    Yes, that's fair.

 3         Q.    Did the Bureau smooth raw adjustment factors in 

 4    1940?

 5         A.    No, sir.

 6         Q.    Did the Bureau presmooth the variances of 

 7    adjustment factors in 1940?

 8         A.    Nope.

 9         Q.    Did the Bureau even calculate adjustment factors 

10    in 1940?

11         A.    The Bureau calculated adjustment factors in the 

12    1950s, I believe, regarding the 1940 census.

13         Q.    Did they -- but they didn't smooth or presmooth 

14    those adjustment factors that they calculated in 1950 for 

15    the 1940 census, is that correct?

16         A.    Well, yes.

17         Q.    My question calls for a yes or no answer.

18         A.    Yes or no.  I mean, the estimates that were 

19    produced for 1940 were produced from the method of 

20    demographic analysis.

21         Q.    So they weren't smoothed or presmoothed, is that 

22    right?

23         A.    Well, that's what I'm fumbling with.  The 

24    demographers that produced these go through enormous labor 

25    to smooth various components of them, yes.  And --
                                                              669

 1         Q.    My question had to do with smoothing adjustment 

 2    factors.

 3         A.    I think I understand your question, and what I'm 

 4    telling you is that the demographers who build up these 

 5    estimates from raw sources of data on births, deaths, et 

 6    cetera, go to some lengths to smooth out the data 

 7    appropriately.

 8         Q.    Did they presmooth any variances?

 9         A.    Ah, gee, I wasn't there then.  I doubt that they 

10    did.

11         Q.    You weren't there, so you are really not 

12    cognizant of what happened in the 1950s, is that correct, on 

13    this process that you are just describing right now?

14         A.    I doubt that they smoothed variances.

15         Q.    Now, you testified, I believe it was earlier 

16    today, that in 1990, the dates for the census operation, the 

17    traditional enumeration and the post-enumeration survey were 

18    separated so that -- these were your words -- the 

19    enumerators wouldn't be tripping over each other.  Is that 

20    correct?

21         A.    Yes, sir.

22         Q.    The census was taken in April of 1990, is that 

23    right?

24         A.    It got under way in late March.

25         Q.    And the PES interviewing was in July and August 
                                                              670

 1    of the same year, is that correct?

 2         A.    Yes, sir.

 3         Q.    And the difference in time between those two 

 4    operations introduced a bias, didn't it?

 5         A.    Huh?

 6         Q.    Well, couldn't people move between those two 

 7    dates?

 8         A.    Sure.

 9         Q.    And doesn't that create problems for matching, a 

10    problem that had to be dealt with in the PES?

11         A.    Let's put it this way, it creates an issue for 

12    PES design and that issue was dealt with.

13         Q.    You testified this morning that you know of no 

14    evidence that shows that matching error may have been 

15    conservatively estimated by the Bureau; is that correct?

16         A.    That's correct.  

17         Q.    You didn't conduct any studies on this issue on 

18    your own, did you?

19         A.    Nope.

20         Q.    You also testified this morning that the Bureau 

21    looked at alternative methods of imputation, and that all 

22    the alternatives gave the same answer.  Isn't that correct?

23         A.    Well, within a very narrow band, yes.

24         Q.    Oh, so they didn't give the same answer.  There 

25    was a narrow band?
                                                              671

 1         A.    Well, it depends on the question.

 2         Q.    Well, I'm asking what your testimony was, and I 

 3    tried to take very good notes of that testimony.  

 4               Did you say that the Bureau conducted studies of 

 5    alternative methods of imputation and all the alternatives 

 6    gave the same answer?  

 7               MR. SOLOMON:  Your Honor, if Mr. Millet wants to 

 8    ask this witness a question, why doesn't he ask him a 

 9    question?  

10               If you want to have the record read back from 

11    this morning, we can do that too.  But it's been precarious 

12    to ask, Tell me each and every word you said, even five 

13    hours ago.  And I object on that basis.  

14               MR. SOLOMON:  I'd be very happy --

15               THE COURT:  Overruled.  

16               MR. SOLOMON:  Thank you, your Honor.

17               THE WITNESS:  Okay.  The various methods of 

18    imputation lead to the same conclusions about the efficacy 

19    of adjustment.

20         Q.    All right.  I'd like you to take a look at 

21    Exhibit -- Plaintiff's Exhibit 540 A.1.

22         A.    Where is this?

23         Q.    It should be in the binders in front of you, Dr. 

24    Wolter.  I'll be happy to try to find it for you.

25               THE COURT:  I think that's 22, isn't it?  Tab 22? 
                                                              672

 1               MR. MILLET:  No, I don't think so.  

 2               MR. SOLOMON:  What P Study?  

 3               MR. MILLET:  P 1.  

 4               MR. SOLOMON:  No.  

 5               MR. MILLET:  Your Honor, may I approach?

 6               THE COURT:  Yes, sure.

 7               (Pause) 

 8    BY MR. MILLET:

 9         Q.    Do you have P 1 in front of you, Dr. Wolter?

10         A.    Yes, sir.

11         Q.    Take a look at the first page of that.  That's 

12    entitled analysis of reasonable alternatives, is it not?

13         A.    Yes, it is.  This is an analysis of alternative 

14    ways of handling the missing data.

15         Q.    Alternatives for imputation, right?

16         A.    Yes.

17         Q.    Take a look at section 3.4, please.  That is 

18    located on page 15.  Tell me when you've got it.

19         A.    I have it.

20         Q.    That section is entitled, "Effective Reasonable 

21    Alternatives," is it not?

22         A.    Yes.

23         Q.    And I'd like you to take a look at that and tell 

24    me whether that lists the mover alternative treatment as one 

25    of the reasonable alternatives being studied in P 1?
                                                              673

 1         A.    You are at the bottom of page 15?

 2         Q.    And carrying over to the top of page 16.

 3         A.    Yes.

 4         Q.    That lists the mover alternative as one of the 

 5    reasonable alternatives reviewed in P 1, isn't that correct? 

 6         A.    It lists it there.  I don't believe it's 

 7    reasonable, however, and I don't believe that the Census 

 8    Bureau included that in many of its subsequent analyses.

 9         Q.    But it is listed as one of the reasonable 

10    alternatives being studied, is it not?

11         A.    At this particular point in this particular 

12    document, yes.

13         Q.    Take a look at Table 3.4 on page 24.  That table 

14    shows that the production bill system estimate of the 

15    undercount is 2.11 percent, doesn't it?

16         A.    Yes, it does.

17         Q.    And it shows using the mover alternative for 

18    imputation that the undercount would be 1.65 percent, isn't 

19    that correct?

20         A.    Yes, it does.

21         Q.    Dr. Wolter, Mr. Solomon asked you about the 

22    reputation of one of plaintiff's witness, so I'm going to 

23    ask you about one of ours.

24               You'd agree with me, wouldn't you, that Professor 

25    Wachter has a very high reputation in the field of 
                                                              674

 1    statistics and democracy?

 2         A.    Generally, yes.

 3         Q.    Dr. Wolter --

 4         A.    Not necessarily in this particular field, but 

 5    generally, yes.

 6         Q.    Dr. Wolter, you obviously spent a lot of time 

 7    working on the PES, I take it, from your direct testimony 

 8    while you were at the Census Bureau.

 9         A.    Yes.

10         Q.    Would it be fair to say that you put a lot of 

11    time and effort into designing the PES and trying to create 

12    something that would work?  Isn't that fair?

13         A.    That's fair.

14         Q.    In fact, wouldn't it be fair to say that you were 

15    one of the key designers of the PES at the time you left the 

16    Census Bureau?

17         A.    I think it's fair.  I certainly wasn't the only 

18    one.  There were many people collaborating in this process.  

19    It would be inappropriate to single anyone out.  It was a 

20    team effort.

21         Q.    And you referred to post-stratification scheme a 

22    couple times during your direct testimony.  It's your view, 

23    I take it from your direct testimony, that the post- 

24    stratification scheme used by the Bureau in the 1990 PES had 

25    the effect of reducing heterogeneity within the poststrata, 
                                                              675

 1    is that correct?

 2         A.    Yes, that's my general answer.

 3         Q.    And your conclusion is based on your professional 

 4    judgment as an expert in the field of statistics and survey 

 5    sampling, is that right?

 6         A.    I suppose it is.

 7         Q.    Now, Dr. Wolter, in the 1990 PES, the poststrata 

 8    are really the units upon which adjustment factors are 

 9    calculated, is that correct?

10         A.    Yes.

11         Q.    And the poststrata in 1990 were divided into 

12    geographic regions, right?

13         A.    Yes.

14         Q.    Those geographic regions in 1990 corresponded to 

15    the Census Bureau divisions, didn't they?

16         A.    That's fair.  I believe the document we looked at 

17    this morning explains the definition of these poststrata in 

18    detail.

19         Q.    That was probably 566 under tab 24 of your 

20    notebook?  Does that sound right, "Preliminary 

21    Stratification Samples for 1990 Census, Coverage by 

22    Measurement Programs"?

23         A.    566.

24         Q.    Yes, tab 24?

25         A.    Tab 24, yes.
                                                              676

 1         Q.    Dr. Wolter, if the definition of a Census Bureau 

 2    division was changed and that new definition was used for 

 3    purposes of post-stratification in the 1990 PES, that could 

 4    alter the estimated undercount of states, couldn't it?

 5         A.    In some small way, I suppose.

 6         Q.    But you never studied that issue as a member of 

 7    the Special Advisory Panel, did you?

 8         A.    Nope, I sure didn't.

 9         Q.    At the time you made your recommendation to the 

10    Secretary, you never reviewed any studies of undercount 

11    estimates using different geographic groupings of the states 

12    other than was actually used in post-enumeration survey, 

13    isn't that right?

14         A.    I don't think that's quite right.  The Census 

15    Bureau was interested in a brief examination of an 

16    alternative post-stratification.  I believe they, in this 

17    alternative, I believe they eliminated the 1392 poststrata 

18    and simply used states as poststrata, and directly estimated 

19    the undercount within the states.  I believe we saw that as 

20    panel members.

21         Q.    But you based your recommendation on the 

22    adjustment on the post-stratification that was actually used 

23    in the 19906 PES, didn't you?

24         A.    Well, I based my decision on all the evidence I 

25    saw, but the decision was with regard to the PES that was to 
                                                              677

 1    be used for correction purposes.

 2         Q.    And that included the geographic post- 

 3    stratification that we just discussed, is that correct?

 4         A.    The 1392, yes.

 5         Q.    Right.  The 1392?

 6         A.    Yes, sir.

 7         Q.    And the 1392 were grouped by geographic regions 

 8    corresponding to Census Bureau divisions, correct?

 9         A.    Yes.

10               MR. MILLET:  Your Honor, may I approach? 

11               THE COURT:  Sure.

12         Q.    Dr. Wolter, I'm handing you what's been marked 

13    Defendant's Exhibit 201.  

14               MR. MILLET:  Your Honor will find it under your 

15    tab 1 in your binder.

16         Q.    Particularly, Dr. Wolter, I'd like to direct your 

17    attention to the last page of that document.  

18               MR. SOLOMON:  Your Honor, I'd like to object to 

19    the use of this document.  The document is dated August 29, 

20    1991.  

21               MR. MILLET:  I haven't even stated the question 

22    yet, your Honor.

23               THE COURT:  Let me hear the question.  

24               MR. SOLOMON:  Very well.  

25    BY MR. MILLET:
                                                              678

 1         Q.    Have you got the last page, Dr. Wolter?

 2         A.    The last page --

 3         Q.    I want you to look at the last page.

 4               THE COURT:  Map?  

 5               MR. MILLET:  Correct, your Honor, the map?

 6         A.    Okay, I have it.

 7         Q.    There's nothing illogical about the grouping of 

 8    states there shown for post-stratification purposes, is 

 9    there, Dr. Wolter?  

10               MR. SOLOMON:  Your Honor, I object to the 

11    question.  And I object to the use of this document.  This 

12    was after Dr. Wolter rendered his recommendation.  It was 

13    after the Secretary's opinion.  The government objected this 

14    morning to the use of a document with this witness that 

15    post-dated the Secretary's opinion, and I withdrew the offer 

16    of the document.  It is a two-way street.  

17               MR. MILLET:  Your Honor, if I may, I'm using this 

18    document solely for illustrative purposes at this point, and 

19    I'm solely using the map, and I'm not using the document for 

20    any other purposes.

21               THE COURT:  Sounds reasonable.  I'll permit it.

22         Q.    Dr. Wolter, let me restate my question because I 

23    think I may have forgotten it at this point.

24         A.    Okay.

25         Q.    I'd like to take a look at the map contained on 
                                                              679

 1    the last page of Defendant's Exhibit 201.

 2               Now, my question to you, Dr. Wolter, is that 

 3    there is nothing illogical about that geographic grouping of 

 4    states for purposes of post-stratification, is there?

 5               (Pause)

 6         A.    Well, on the one hand, it has some face validity, 

 7    and, on the other hand, there are some questions that 

 8    deserve further study, like:  Do you put Hawaii with 

 9    California? -- for example.  Do you put Alaska with Utah?  

10    Do you put Oklahoma with West Virginia?

11               But that said, it has some face validity.

12               MR. MILLET:  I offer defendants 201 for 

13    illustrative purposes only. 

14               MR. SOLOMON:  Your Honor --

15               THE COURT:  Just a map.  

16               MR. MILLET:  I confine it to the map.

17               THE COURT:  Just the map.  For illustrative 

18    purposes only.  

19               MR. MILLET:  I would note that one exhibit of 

20    plaintiffs' 706 was received on Monday for illustrative 

21    purposes only 

22               MR. SOLOMON:  Are we going to detach the map from 

23    the document and mark it separately?  

24               MR. MILLET:  I'll be happy to do that.  

25               MR. SOLOMON:  Okay, no objection.
                                                              680

 1               THE COURT:  Okay.  Let's do it that way.

 2               (Defendants' PX 201 for identification was 

 3    received in evidence) 

 4    BY MR. MILLET:

 5         Q.    Dr. Wolter, isn't it true that you have no reason 

 6    to suppose that people with the same demographic 

 7    characteristics in West Virginia are more like those in 

 8    Georgia rather than people in Kentucky when it comes to 

 9    capture probability?  Isn't that correct?

10         A.    I'm sorry.  Could you repeat the question?

11         Q.    You have no reason, do you, Dr. Wolter, to 

12    believe that persons with the same demographic 

13    characteristics who happen to live in West Virginia are more 

14    like people with the same demographic characteristics in 

15    Georgia rather than Kentucky when it comes to capture 

16    probability?

17         A.    Well, it's still somewhat of a convoluted 

18    question, but the West Virginia, and Kentucky, in this case, 

19    are in group 6.  Georgia is in group 4.  But yet they are in 

20    the same general geographic neighborhood of one another as 

21    well.

22               As we said this morning, there is residual 

23    heterogeneity, no matter how you divide it up.

24         Q.    My question is whether you have any reason to 

25    believe that persons with the same demographic 
                                                              681

 1    characteristics in West Virginia are more like those who 

 2    live in Georgia with the same demographic characteristics 

 3    rather than people in Kentucky with the same demographic 

 4    characteristics when it comes to capture probability?

 5         A.    I -- I don't necessarily have reason to feel one 

 6    way or another, nor do I think it matters very much 

 7    regarding the efficacy of the matter before us.

 8         Q.    The ideal for a PES would be that everyone in the 

 9    same poststratum had the similar capture probability, isn't 

10    that right?

11         A.    That is the ideal.

12         Q.    And I take it from your testimony today that you 

13    would agree that there's always going to be some 

14    imperfection in any post-stratification scheme, isn't that 

15    right?

16         A.    That's correct.

17         Q.    In fact, wouldn't you agree that there's no way 

18    to test whether everyone in the same poststratum has the 

19    same capture probability?

20         A.    I think that's a fair statement, yes.

21         Q.    In fact, wouldn't you agree there's no way on 

22    earth to conclusively test that everyone in a particular 

23    poststratum has the same capture probability?

24         A.    I think there's no way on earth, nor do I think 

25    it matters, for the reasons that I stated this morning,  
                                                              682

 1    this process still leads to improvement in spite of the 

 2    residual heterogeneity.

 3         Q.    Would you agree that the main statistical purpose 

 4    of the census is the production of small area data?

 5         A.    I have mixed feelings about that.  In one sense, 

 6    I think the main purpose of the census is the Constitutional 

 7    purpose.

 8         Q.    I asked you the main statistical purpose of the 

 9    census.  Wouldn't you agree that the main statistical 

10    purpose of the census is the production of small area data?

11         A.    My answer is the same.  I have mixed feelings 

12    about this.  Clearly, there are thousands and thousands and 

13    thousands of very important uses of census data.  Some of 

14    them are at small area levels.  But the pre-eminent purpose 

15    of the census, both statistically and otherwise, is the 

16    Constitutional purpose.

17         Q.    Do you remember being deposed in 1989 in this 

18    case?

19         A.    Yes, I do.

20         Q.    Do you remember being asked during that 1989 

21    deposition the following question.

22               "Q.    Do you agree that the main statistical 

23    purpose of the census is the production of small area 

24    data?"

25         A.    No, I don't.
                                                              683

 1         Q.    Do you remember giving this answer to that 

 2    question:  

 3               "Yes.  I'm inclined to agree that the main 

 4    statistical -- and, if possible, on the record underscore 

 5    the word 'statistical' -- for the purpose of the census is 

 6    small area data"?

 7         A.    Like I said, I don't remember the question, let 

 8    alone the answer.  

 9               MR. MILLET:  Your Honor, may I approach?

10               THE COURT:  Yes.

11         Q.    Dr. Wolter, I'd like you to look at page 266 of 

12    the transcript of your first deposition in the case 

13    beginning on lines 12, down to about the end of that page.

14               Let me ask you if that refreshes your 

15    recollection as to whether or not you stated in that 

16    deposition that you agree the main statistical purpose of 

17    the census is the production of small area data?

18         A.    Let me try and explain this.

19         Q.    No.  My question, Dr. Wolter, is if this 

20    refreshes your recollection as to whether you agree that the 

21    main statistical purpose of the census is the production of 

22    small area data?

23         A.    This refreshes my recollection that I said that, 

24    and that --

25         Q.    Thank you, Dr. Wolter.
                                                              684

 1         A.    -- and that the word "statistical" was put in 

 2    quotes.

 3         Q.    Put in quotes and underscored, I think you 

 4    wanted?

 5         A.    And underscored, right.  That has a very special 

 6    meaning in my mind.

 7         Q.    Wouldn't you also agree that the central issue 

 8    for adjustment is whether -- no, strike that.

 9               Dr. Wolter, in your experience as a former 

10    official of the Census Bureau, senior government officials 

11    usually don't write their own documents, do they?

12         A.    That's true, some of them can't.

13               (Laughter)

14         A.    Some of them can't think either.

15         Q.    Dr. Wolter, isn't it axiomatic that the greatest 

16    difficulties in measuring the undercount are encountered in 

17    the same areas where the census itself encounters great 

18    difficulties?

19         A.    On balance, that is basically true.  You may find 

20    exceptions, but that tends to be our general belief.

21         Q.    Now, demographic analysis was part of your 

22    overall consideration of adjustment for the 1990 Decennial 

23    Census, isn't that right?

24         A.    Yes, it was.

25         Q.    However, you believe that demographic analysis 
                                                              685

 1    has some shortcomings, don't you?

 2         A.    Well, yes, I do.

 3         Q.    One of those shortcomings is undocumented 

 4    immigration, isn't that right?

 5         A.    Yes, sir.

 6         Q.    And another shortcoming is that there are 

 7    components of the population that aren't covered by birth 

 8    and death registration, isn't that correct?

 9         A.    Yes, that's --

10         Q.    Demographic analysis --

11         A.    I'm not sure that that's the main issue, but, 

12    sure.

13         Q.    The demographic analysis tries to make an 

14    allowance for under-registration of birth records, doesn't 

15    it?

16         A.    Yes, it does.

17         Q.    And you have some concerns and reservations about 

18    that process, don't you?

19         A.    Yes, I do.

20         Q.    Now, the Bureau calculated confidence intervals 

21    for its demographic analysis estimates, isn't that correct?

22         A.    I believe it did, yes.

23         Q.    And some of that work even started while you were 

24    still at the Bureau, didn't it?

25         A.    Yes.  I urged its beginning.
                                                              686

 1         Q.    And those confidence intervals reflect judgments 

 2    made by the Bureau's demographers, don't they?

 3         A.    Judgments by the Bureau's demographers and 

 4    others, I believe.

 5         Q.    But they are not the kind of confidence intervals 

 6    which you as a statistician use, isn't that correct?

 7         A.    In a narrow sense, they are not quite the same, 

 8    but, nonetheless, carrying out that work, in trying to 

 9    understand the structure of strength and error in 

10    demographic analysis is far better than not doing the work 

11    at all.

12         Q.    Isn't it true, Dr. Wolter, they are judgmental 

13    confidence intervals and not statistical confidence 

14    intervals?

15         A.    I think that's probably a fair statement.

16         Q.    Now, you referred in your direct examination, I 

17    believe it was this morning, to a paper that you wrote with 

18    one of your former colleagues at the Bureau, Beverly Causey, 

19    is that right?

20         A.    Yes, sir.

21         Q.    And that was a paper on evaluating the accuracy 

22    of proofing small areas to an adjustment.  

23         A.    (No response)

24         Q.    I'm sorry.  Is that a yes?

25         A.    Yes.  We spoke about that paper.
                                                              687

 1         Q.    I only say that because the court reporter has to 

 2    take down the words.           

 3               MR. MILLET:  May I approach, your Honor?

 4               THE COURT:  Sure.  

 5               MR. MILLET:  Thank you.

 6         Q.    Dr. Wolter, I've handed you what has been marked 

 7    as Defendant's Exhibit 202.  

 8               MR. MILLET:  Your Honor, it's fully also 

 9    contained in Plaintiff's Exhibit 195.  So you'll find a copy 

10    of the excerpt I've been referring to under tab 2 of your 

11    binder.

12         Q.    Is this the paper that you were referring to with 

13    Dr. Causey a moment ago, Dr. Wolter?

14         A.    Yes.  This was a prepublication draft.

15         Q.    Now, this is included in Appendix G to the main 

16    report with you and your three colleagues on the panel to 

17    the Secretary, is that right?

18         A.    That's correct.

19         Q.    This paper is not based on 1990 Census data, is 

20    it?

21         A.    Well, it was written before 1990.

22         Q.    That would have been pressure for you to be able 

23    to use 1990 data?

24         A.    That would have been difficult.

25         Q.    Isn't it true that you based this paper 
                                                              688

 1    exclusively on 1980 census data, Dr. Wolter?

 2         A.    I wouldn't say I based the paper exclusively.  

 3    Whatever --

 4         Q.    That was your data base, wasn't it, 1980?

 5         A.    Whatever empirical data appears in here is data 

 6    from the 1980 census.

 7         Q.    Now, you calculated what you referred to as break 

 8    even CV. in this paper, is that correct?

 9         A.    Yes, sir, among other things.

10         Q.    Aren't the break even CV. one of the, if I can 

11    use the term, innovations that you believe this paper brings 

12    to the scholarship in the field?

13         A.    Well, sure, that's one reason we sought to 

14    publish the paper.

15         Q.    And you calculated the break even CV on table 4, 

16    page 22 in this paper, is that correct?

17         A.    Yes.  Again, using 1980 data, these -- the table 

18    reveals some of the results of our calculations.

19         Q.    That's an important qualification, Dr. Wolter.  

20    That's the point of my next question.

21               In fact, table 4 can't be used for any purpose in 

22    1990, can it?

23         A.    Can't be used for any purpose in 1990.  No, I 

24    don't think that's quite right.

25         Q.    You don't think that's right?
                                                              689

 1         A.    I don't think that's quite right, no.  

 2               I think from this table, and from other work in 

 3    this paper generally, we learn that a strict link down 

 4    process -- if I can use that technical term that was used 

 5    this morning -- that is, the allocation of the undercount 

 6    from broad geographic areas to blocks within the broad area 

 7    is better accomplished by way of the kind of synthetic 

 8    method utilized in 1990 than by another simpler method, 

 9    simpler but very much related method, called across the 

10    board.  And that finding is pretty striking in this paper as 

11    well as in our work, so striking that I think it's certainly 

12    generalizable from 1980 data to the 1990 Census itself.

13         Q.    Dr. Wolter, do you remember your second 

14    deposition in this case, beginning of April of this year?

15         A.    Yes.

16         Q.    Do you remember testifying in that deposition 

17    that table 4 numbers should not be used for any purpose in 

18    1990?  Do you remember that testimony?

19         A.    No.

20         Q.    Let me direct counsel to page 862.  

21               MR. MILLET:  Your Honor, may I approach?  

22               (Pause) 

23    BY MR. MILLET:

24         Q.    Dr. Wolter, let me direct your attention to page 

25    862 of your deposition, lines 8 through 9.  Did you or did 
                                                              690

 1    you not testify in your deposition in April of this year 

 2    that the table 4 numbers should not be used for any purpose 

 3    in 1990?

 4         A.    Evidently, that's what I said in response to a 

 5    narrow question.

 6               (Continued on next page) 

 7    

 8    

 9    

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              691

 1    BY MR. MILLET:

 2         Q.    Dr. Wolter, you found the P studies to be 

 3    accurate for the 1990 PES, is that correct?

 4         A.    The P studies provided adequate quality checks to 

 5    form good conclusions about the efficacy of correction in 

 6    1990.

 7         Q.    They provided useful information to you, is that 
                                                                  

 8    right?

 9         A.    Yes, sir.

10         Q.    And it is your view that the P studies were 

11    conducted in a professional manner, is that correct?

12         A.    Generally, yes.

13         Q.    But you didn't check any --

14         A.    I'm not sure they were all cost-effective, but I 

15    think they were generally performed in a professional 

16    manner, yes.

17         Q.    In fact, you suggested that some of them should 

18    be a little more cost-effective than they actually ended up 

19    being by using smaller samples, didn't you?

20         A.    I think I did indeed.

21         Q.    But you didn't check any of the P studies for 

22    errors, did you?

23         A.    Check the P studies for errors?

24         Q.    Correct.

25         A.    Well, the P studies are quality checks on the PES 
                                                              692

 1    which, in turn, evaluates the original enumeration so --

 2         Q.    So the answer to my question is yes or no?

 3         A.    P studies are a third order effect.  They were 

 4    conducted by people I had confidence in and, no, I didn't 

 5    dig in and reconfirm this third order effect, no.

 6         Q.    You would agree with the proposition that 

 7    sampling and nonsampling error in the PES should be 

 8    controlled to the level of one to two percent, isn't that 

 9    right?

10         A.    When we say controlling to one --  one or what?

11         Q.    One or two percent.

12         A.    --  one or two percent, that, that is referring 

13    to work that was done on the design of the PES.

14               In other words, the sampling  -- 

15         Q.    Dr. Wolter, I didn't ask you about the design, I 

16    asked you whether you agreed with the proposition that 

17    sampling and nonsampling errors should be controlled in PES 

18    to one or two percent.  That's a yes or no question.

19         A.    We sought to design the program to achieve those 

20    low levels, yes.

21         Q.    So the answer is yes?

22         A.    Yes.

23         Q.    And I take it from your testimony this morning 

24    that you are familiar with the Bureau's total error model, 

25    right?
                                                              693

 1         A.    Yes.

 2         Q.    In fact, you relied on the Bureau's total error 

 3    model to synthesize the sources of nonsampling error, is 

 4    that correct?

 5         A.    Yes, sir.

 6         Q.    And isn't it your view that the total error model 

 7    was supposed to include all sources of nonsampling error 

 8    provided the Bureau did its work correctly?  Isn't that 

 9    right?

10         A.    Well, all relevant sources, yes.

11         Q.    But the Bureau's total error model didn't take 

12    into account late-late census data, did it?

13         A.    I believe that the Bureau studied late-late 

14    census data in one of the P studies.

15         Q.    That's not my question.  My question is the 

16    Bureau's total error model did not take into account 

17    late-late census data, did it?

18         A.    I'm not sure that it did in a direct way, but, 

19    then again, if it was important to do so, I believe the 

20    census staff would have done so.

21         Q.    So your answer is yes or no or you couldn't know?

22         A.    What was the question?

23         Q.    I will read the question again.

24               The Bureau's total error model did not take into 

25    account late-late census data, did it?
                                                              694

 1         A.    It may not have.  On the other hand, what I am 

 2    saying is that if it was important to do so, I believe they 

 3    would have done so.

 4         Q.    But as you sit here today, you don't know whether 

 5    it did or didn't?

 6         A.    We earlier today looked at the results of the 

 7    total error analysis and late-late census data, as you are 

 8    calling it, was not reflected in that total error analysis.

 9         Q.    Thank you.

10         A.    On the other hand, you got to remember 

11    late-late-late census data may be pretty much garbage and 

12    perhaps that's what they had in their mind in not including 

13    it.

14         Q.    But you don't know, do you?

15         A.    I don't know what?

16         Q.    You don't know that they considered it to be 

17    garbage, do you?

18         A.    I have a pretty good idea what many thought about 

19    it, yes.

20         Q.    Dr. Wolter, you testified to the computer 

21    matching process for the post-enumeration survey a little 

22    earlier today, is that right?

23         A.    Yes, sir.

24         Q.    And it's your view that there was a layering of 

25    computer matching and human matching in the PES, is that 
                                                              695

 1    right?

 2         A.    That's correct.

 3         Q.    But the computer matching was able to match 

 4    accurately about 75 percent of the PES.  I believe that's 

 5    what you testified to, is that right?

 6         A.    I, I believe that's correct, yes.

 7         Q.    In fact, wasn't the accuracy of the computer 

 8    matching process one of the things that led you to believe 

 9    that an adjustment could be done through a post-enumeration 

10    survey?  Isn't that right?

11         A.    No, that's, that's actually very slightly wrong.  

12    Let me explain what I mean.

13         Q.    You don't believe that the computer matching 

14    process was one of the things that led you to believe that 

15    an adjustment could be done for a post-enumeration survey?

16         A.    It, it, it among other factors led us to believe 

17    that the post-enumeration survey would, in effect, work 

18    well.

19               Let me explain what is slightly wrong about your 

20    statement.

21         Q.    Well, I want to know --

22         A.    Computer matching --

23         Q.    Dr. Wolter, I want to know whether you believe 

24    whether the accuracy of the matching process was one of the 

25    factors that led you to believe that an adjustment could be 
                                                              696

 1    done.  It's a yes or no question.  

 2               MR. SOLOMON:  Your Honor, I do object to the 

 3    continual interruption of the witness.  The witness is 

 4    being --  

 5               THE COURT:  I think the question does admit of a 

 6    yes/no answer.

 7         A.    May I have the question again?

 8         Q.    The question, Dr. Wolter, is whether the accuracy 

 9    of the computer matching was one of the things which led you 

10    to believe that an adjustment could be done through 

11    post-enumeration survey.

12         A.    The accuracy of the computer matching?

13         Q.    Correct.

14         A.    Is, in part, one of the factors that led us to 

15    the conclusions, but it certainly wasn't a dominant factor.

16         Q.    I didn't mean to imply that it was an exclusive 

17    factor or there weren't other factors, I just wanted to know 

18    whether it was one of them.

19         A.    Okay.  Fair enough.

20         Q.    Now, isn't it fair to say, Dr. Wolter, if there 

21    were an error in the computer matching program that was to 

22    have inflated the net national undercount by over one 

23    million people, that that factor in and of itself really 

24    wouldn't be enough to affect your opinion on adjustment in 

25    1990, isn't that right?
                                                              697

 1         A.    It depends on --  are you saying there is an 

 2    error?

 3         Q.    I am supposing to you that there is, indeed, an 

 4    error in the computer matching process and that process 

 5    resulted in a net national undercount that was one million 

 6    persons too high, and my question to you is, that fact in 

 7    and of itself is not necessarily enough to get you to affect 

 8    your opinion regarding adjustment in 1990?

 9         A.    In and of itself, no.  We need to know whether 

10    the error was caught in the other layers of the matching 

11    process, we need to know how the errors distribute itself 

12    across the country.  Stopping as you did doesn't tell us 

13    enough.

14         Q.    But that fact certainly would affect your opinion 

15    regarding the reliability of the computer matching process, 

16    wouldn't it?

17         A.    It would, it's would affect my opinion of, of 

18    that particular implementation of the computer matching 

19    process.

20         Q.    So if the computer had created an extra million 

21    people in the undercount, that alone would not be enough to 

22    affect your opinion on adjustment?

23         A.    Sitting here in a vacuum without, without the 

24    evidence and without a careful examination, I, I can't say 

25    that it would.
                                                              698

 1         Q.    But that would mean an error of that magnitude by 

 2    the computer, that would mean that the adjustment that you 

 3    recommended to the Secretary would have been one million 

 4    people too large, isn't that right?

 5         A.    Oh, but that's what I'm saying, I'm not sure that 

 6    it would.

 7               If the computer matchers did make a mistake, for 

 8    heaven's sake, there are these other layers of review that I 

 9    talked about that may well have eliminated this problem.

10         Q.    Dr. Wolter, I would like you to assume that the 

11    other layers of review did not eliminate that problem 

12    created by the computer.

13         A.    Okay.

14         Q.    Wouldn't that mean that the adjustment that you 

15    proposed to the Secretary in July of 1991 would have been 

16    too large?  Isn't that correct?

17         A.    Too large in absolute terms, probably, yes.

18         Q.    Now, you and your colleagues --

19         A.    Maybe not in distributable terms.

20         Q.    I understand the distinction.

21               You and your three colleagues on the special 

22    advisory panel reported to the Secretary that the Census 

23    Bureau found 13 unweighted fabrications in the 

24    post-enumeration survey, didn't you?  

25               THE COURT:  Unweighted?  
                                                              699

 1               MR. MILLET:  Unweighted.

 2         A.    That simply means though found 13 fabricated 

 3    households or people in the entire PES.

 4         Q.    Answer the question.  

 5               THE COURT:  Out of 400,000 people? 

 6               THE WITNESS:  Yes.  Yes, sir.

 7         Q.    The answer to the question is yes?

 8         A.    Out of 400,000 people, yes.

 9         Q.    You reported to the Secretary that the Census 

10    Bureau found 13 fabrications?

11         A.    Well, yes.  That's what the Census Bureau found.

12         Q.    And you didn't find that --

13         A.    I didn't find them, the Census Bureau found them.

14         Q.    I understand that.  That was my question.

15               Now my answer is, you didn't find that to be 

16    unusual, Dr. Wolter, out of a sample that large the Census 

17    Bureau would have found only 13 unweighted fabrications?

18         A.    In the history of survey sampling, that is very 

19    unusual.  The Census Bureau had a marvelous job of 

20    controlling the fabrication level to very low levels.

21               Given the in-production quality checks and the 

22    massive effort that the Bureau put into controlling this to 

23    that low level, I'm not surprised.

24         Q.    So because of the quality of the PES and the 

25    quality of the staff, many of which I understand from your 
                                                              700

 1    direct testimony you hired, you didn't find it unusual that 

 2    the Bureau would report out of a sample survey this large 

 3    only 13 fabrications, is that right?

 4         A.    I thought I just answered the question.

 5         Q.    Well, it is a little different question, but you 

 6    will pardon for me being repetitive.

 7         A.    Well, again, in the history of survey sampling, 

 8    this is a surprisingly low number.

 9         Q.    But not in this case?

10         A.    But not in this case given all the effort that 

11    went into it.

12         Q.    Didn't you also report to the Secretary that 

13    those 13 fabricated weighted up to a national level added up 

14    to 64,000 people?

15         A.    I don't recall the number, no.  I recall the 13.  

16    I don't recall the weighted number.

17               Weighted simply means we are projecting from the 

18    sample up to the universe again.  That's all the word 

19    "weighting" means.

20         Q.    Dr. Wolter, I am showing you what has been marked 

21    as Defendant's Exhibit 203.  

22               MR. MILLET:  Your Honor, it is under tab 3 of the 

23    cross-examination exhibits.  It is also contained in 

24    Plaintiff's Exhibit 195.

25               (Handing to the witness)
                                                              701

 1               (Pause)

 2         Q.    Dr. Wolter, I would like you to take a look at 

 3    page --  first of all, Dr. Wolter, that is appendix C to the 

 4    report by you and your three colleagues to the Secretary on 

 5    adjustment, isn't it?

 6         A.    Yes.

 7         Q.    Will you turn to page 16, please.

 8               Do you indicate to the Secretary on page 16 that 

 9    the 13 fabrications found in P5a had a weighted sum of 

10    64,667 persons?

11         A.    Evidently we did report that, yes.

12         Q.    So changing the status of this 13 people in one 

13    of the evaluation studies can change the estimated bias in 

14    the PES by over 64,000 people, is that correct?

15         A.    That evidently is the way it comes out.

16         Q.    Didn't the Bureau --

17         A.    You have to understand, this is 64,000 of 250 

18    million.  That is pretty small potatoes.

19         Q.    And it is only 13 people out of the evaluation 

20    studies, isn't that right?

21         A.    Well, 13 people --

22         Q.    Thirteen cases?

23         A.    Thirteen cases out of the PES, not out of the 

24    quality checks.

25         Q.    Didn't the Bureau conduct other studies of 
                                                              702

 1    fabrications that resulted in higher estimates of 

 2    fabrications than the 13 that you report here to the 

 3    Secretary?

 4         A.    Well, yes, but --

 5         Q.    Well, did the --

 6         A.    The other studies weren't utilized, again, in the 

 7    total error analysis.  I believe this was the input, if I 

 8    recall correctly, I may not, but as I recall, this is, this 

 9    is what the Bureau felt was their best evidence.

10         Q.    Didn't the Bureau report in a P study that the 

11    fabrication rate ranged from 2.1 percent to 8.79 percent?

12         A.    Well, I don't recall that offhand.

13         Q.    You don't recall that?

14         A.    No.

15               Again, I think this 13, according to the people 

16    that the panel talked with at the Bureau, represents the 

17    Bureau's best and final judgment about fabrication.  

18               MR. MILLET:  Your Honor, may I have a moment with 

19    your indulgence.

20               (Pause)

21         Q.    Dr. Wolter, did you advise the Secretary of any 

22    limitations to P5a?

23         A.    Well, as you pointed me to the 64,000, I noted 

24    that in the earlier line we dismissed project P5a, although 

25    at the moment I can't recall exactly what P5a was.
                                                              703

 1         Q.    Did you advise the Secretary that P5a was not 

 2    designed specifically to detect fabrication?

 3         A.    Perhaps you will point me to the passage where I 

 4    recommended that.  I don't recall it offhand.

 5         Q.    That's why I am asking the question.

 6               Let me point you to P5a.

 7               I promised you I would give it back.

 8               (Handing to the witness)

 9               Take a look at page 9 of P5a, the last paragraph.  

10    Doesn't that indicate that P5a was not designed to detect 

11    fabrication?

12         A.    What am I looking at, first of all?

13         Q.    I am asking you to take a look at Plaintiff's 

14    Exhibit 540 A, I believe it is 5.A.

15         A.    Okay.

16         Q.    I am asking you to look at page 9.

17               First of all, that is project P5a, is it not?

18         A.    Yes, it is, 

19               MR. ZIMROTH:  Excuse me, your Honor.  This is 

20    extraordinarily misleading cross-examination.  Your Honor, 

21    we went through this with Professor Ericksen.  

22               The report is dated June 20.  P5a that he has 

23    asked Dr. Wolter to look at is dated in July something, 

24    whatever the date is there, and we went through the same 

25    thing with Professor Ericksen.  It is just misleading.  
                                                              704

 1               MR. MILLET:  First of all, your Honor, I would 

 2    like to ask who is going to be objecting here.  I thought 

 3    the witness was being examined by Mr. Solomon.  I do take 

 4    offense to being double-teamed.  

 5               THE COURT:  Oh, you can handle it.  

 6               MR. MILLET:  They only got us outnumbered six to 

 7    one.  

 8               THE COURT:  I think we aired our grievances.  

 9    Let's go ahead.  

10    BY MR. MILLET:

11         Q.    Dr. Wolter, my question is P5a was not designed 

12    specifically to detect fabrications, was it?

13         A.    May I have a moment to read a bit of this?

14         Q.    Sure.

15               (Pause)

16         A.    Well, okay.  This is telling us that the 

17    evaluation P5a was designed to evaluate fabrication in the 

18    P-sample, period.

19         Q.    I'm sorry, was designed to evaluate fabrication 

20    in the P-sample?

21         A.    That's, that's what it says in the first line of 

22    paragraph 2 on page 9.

23         Q.    Directing your attention to the second paragraph, 

24    Dr. Wolter, does it not indicate that the data for the study 

25    were collected in the evaluation followup which was not 
                                                              705

 1    designed specifically to detect fabrication?

 2         A.    Yes, it says that, too.  

 3               This evaluation followup, as it is called, was a 

 4    general purpose vehicle for carrying out a number of the 

 5    quality checks, post-production quality checks, it was 

 6    designed with multiple purposes in mind.  So on the one hand 

 7    they are telling us here it wasn't specifically designed to 

 8    detect fabrication, but in the earlier line that I referred 

 9    you to they are also telling us that this evaluation was 

10    designed to estimate fabrication in the P-sampling, residual 

11    fabrication, if any, in the P-sampling.

12         Q.    You advised the Secretary with your three 

13    colleagues that the number of unresolved cases in the 

14    P-sample was 1.5 percent and.9 percent in the E-sample, did 

15    you not?

16         A.    That sounds close.  Again, I don't remember the 

17    numbers exactly, but the order of magnitude sounds right.

18         Q.    And unresolved cases would be handled through 

19    imputation, isn't that correct?

20         A.    Yes, sir.

21         Q.    Didn't the Bureau report that the number of 

22    unresolved cases for the P-sample and the E-sample was 

23    actually higher than that?

24         A.    Gee, I don't recall.

25         Q.    You don't recall?
                                                              706

 1         A.    No.

 2         Q.    But you are familiar with imputation, aren't you?

 3         A.    Yes, sir.

 4         Q.    You are also familiar with the Bureau's 

 5    evaluation followup procedures?  In fact, you just referred 

 6    to it in an aspect there in P 5, did you not?

 7         A.    Yes.

 8         Q.    Wouldn't you expect that inputed cases that were 

 9    matched in the evaluation followup would have a higher 

10    probability of a correct inputed match than cases that were 

11    not matched in the evaluation followup?  Isn't that correct?

12         A.    If the, if the imputation was working ideally, 

13    yes.  But as we established earlier, reasonable imputations, 

14    except for the one unreasonable one that you called my 

15    attention to earlier, all lead to similar conclusions about 

16    the efficacy of census correction.

17         Q.    Well, wouldn't you expect that correlation to be 

18    present if the methodology for imputation was a sound 

19    methodology?  Wouldn't you expect to see that finding?

20         A.    If the methodology was a sound methodology, I 

21    would expect that the efficacy of census correction wouldn't 

22    matter, right.

23         Q.    That's not my question.

24               My question guess to imputation and the 

25    probability of a correct inputed match for cases that were 
                                                              707

 1    actually matched in evaluation followup.

 2               If the imputation methodology was operating 

 3    correctly, wouldn't you expect to see that correlation 

 4    between matched probability in imputation and actual 

 5    matching in the evaluation followup?

 6         A.    I'm loosing the sense of your question.

 7         Q.    Let me try it again.

 8               Wouldn't you expect that inputed cases that were 

 9    matched in evaluation followup, the first half, that --

10         A.    Inputed cases that were matched in the evaluation 

11    followup?

12         Q.    Right.  Cases that originally went to imputation 

13    that then in an evaluation followup were matched, wouldn't 

14    you expect that those cases would have a higher probability 

15    of a correct inputed match than cases that were not matched 

16    in the evaluation followup?

17         A.    Perhaps.  Perhaps.

18               On the other hand, I think you have to keep in 

19    mind that the evaluation followup is being conducted a year 

20    later.

21         Q.    Dr. Wolter, do you remember testifying in your 

22    deposition that if the methodology that is being used for 

23    imputation is a reasonably sound one, you would expect to 

24    see the kind of correlation that I just asked you about?

25         A.    Perhaps.
                                                              708

 1         Q.    Well, let me read you this question and answer 

 2    and see if it refreshes your recollection.  

 3               MR. SOLOMON:  What page?  

 4               MR. MILLET:  I am reading from page 688, line 8.

 5         Q.    "Q.    Would you ordinarily expect that the 

 6    persons that were matched in the EFU, evaluation followup, 

 7    would have a higher probability of a correct inputed match 

 8    than those that were not matched in the evaluation followup?

 9               "A.    Well, if the methodology that is being 

10    used for imputation is a reasonably sound one, then I would 

11    expect to see that finding, which evidently I see here," and 

12    you are referring to a document.

13               Does that refresh your recollection as to whether 

14    you expect to see that correlation?

15         A.    Well, I suppose it does.  I think that's more or 

16    less what I just said.

17         Q.    You and your three colleagues reported to the 

18    Secretary that the proxy interview rate was 2.4 percent, is 

19    that correct?

20         A.    That sounds close.  I don't recall the number 

21    exactly.

22         Q.    It sounds about right to you?

23         A.    Yes.

24         Q.    Proxy interviews are interviews of nonhousehold 

25    members, aren't they?
                                                              709

 1         A.    Yes.

 2         Q.    And those are generally considered less reliable 

 3    than interviews with household members, isn't that correct?

 4         A.    Generally.  It depends on who the proxy 

 5    respondent is.

 6         Q.    Didn't the Bureau conduct a study showing that 

 7    the proxy interview rate for the PES was actually 4.25 

 8    percent?

 9         A.    I don't recall.

10         Q.    Correlation bias is not measured in any of the 

11    P studies, is it?

12         A.    My good friend and colleague at the Bureau, Bill 

13    Bell, has, has done everything possible to try to come to 

14    some understanding of correlation bias.

15         Q.    That is not what I asked, you Dr. Wolter.  I 

16    asked you whether it was measured in any of the P studies.  

17    Was correlation bias measured in any of the P studies?

18         A.    The work I'm referring to by Bell is, is part of 

19    the quality checks that were done on the PES.  Although at 

20    the moment I have to confess I don't recall whether it was 

21    designated as a P study or not, but it was certainly part of 

22    the general work that was done to check the quality of the 

23    work.

24         Q.    Well, isn't it your view, Dr. Wolter, that 

25    correlation bias is estimated through experience in 
                                                              710

 1    conducting decennial census over the years and a result of 

 2    demographic analysis?

 3         A.    Yes and, in fact, Bell's work was using 

 4    demographic analysis.

 5         Q.    Dr. Wolter, if corrections for nonsampling error 

 6    reduced the net national undercount by four million, 

 7    wouldn't that mean that the adjustment you recommended to 

 8    the Secretary in July 1991 at a national level would have 

 9    gone too far?

10         A.    If the estimated undercount was four million too 

11    high?

12         Q.    No.  That the estimated undercount was reduced by 

13    four million due to nonsampling error.

14         A.    Yes.  Well, that's the same statement, really.  

15    It says that production estimate, then, would have been four 

16    million too high.

17         Q.    Correct.

18               Wouldn't that mean that the adjustment that you 

19    recommended to the Secretary would have gone too far?

20         A.    It would have gone too far in absolute terms, 

21    yes.

22         Q.    Isn't it common, Dr. Wolter, in statistics to 

23    find situations where a given problem will be analyzed 

24    differently by different statisticians?

25         A.    That, that, that is common, yes.
                                                              711

 1         Q.    Isn't it also common, Dr. Wolter, for different 

 2    statisticians to use different methodologies and different 

 3    tools to approach statistical problems from different 

 4    perspective?

 5         A.    That's been known to occur, yes.

 6         Q.    You opposed adjustment in 1980 on the model 

 7    offered by Professor Ericksen, right?

 8         A.    In 1980?

 9         Q.    In 1980.

10         A.    Yes.  There were major flaws in 1980.  I opposed 

11    correction at that time, yes.

12         Q.    And wasn't it your view that the suggestion 

13    underlying the model offered by Professor Ericksen in 1980 

14    failed to a degree that rendered them nonusable for 

15    adjustment in 1980 census?  Isn't that correct?

16         A.    I think as a general statement that's correct.  

17    The principal problem with 1980, though, was flawed PES 

18    implementation and data.  That was the basic problem in 

19    1980.

20         Q.    In fact, though, Dr. Wolter, isn't it your view 

21    that no statistician who had access to the data that you had 

22    access to in 1980 could have favored an adjustment of the 

23    1980 decennial census?

24         A.    My censuses, yes, that those who had the intimate 

25    knowledge that we did, the data that we did would be, would 
                                                              712

 1    be led to the same conclusions as I and others were at the 

 2    Census Bureau.

 3         Q.    I believe you referred earlier this morning to 

 4    not being able to discern the true population of the United 

 5    States.

 6               Isn't it your view, Dr. Wolter, that attempting 

 7    to learn the true population of the United States is a 

 8    fundamentally intractable problem?

 9         A.    Right.  We are never going to know truth.

10         Q.    We are never going to know truth?

11         A.    If we did, we would use it.  I'd use it if I knew 

12    it.  But we are not going to --

13         Q.    We would all use it if we knew it?

14         A.    Right.

15         Q.    Given that the true population is not knowable, 

16    determining whether an adjustment moves the estimate of the 

17    population closer to the truth requires the exercise of 

18    professional judgment, doesn't it, Dr. Wolter?

19         A.    I, I think there is an element of professional 

20    judgment as well as a good element of real data, quality 

21    checking of the kind that I have tried to describe.

22         Q.    But there is an element of professional judgment, 

23    correct?

24         A.    I think as a general statement, yes.

25         Q.    The adjustment procedures for 1990 involved the 
                                                              713

 1    use of statistical models, correct, at various levels; the 

 2    imputation, smoothing?

 3         A.    Part of the process could be viewed as based on a 

 4    statistical model, yes.

 5         Q.    The smoothing was a model, wasn't it?

 6         A.    Could be viewed as that, yes.

 7         Q.    And 1990 models used assumptions, didn't it?

 8         A.    If, if we accept that there is a model, yes, then 

 9    --  a model, incidentally, is a kind of mathematical 

10    statistical simplification, it's a representation of how the 

11    world behaves in some way.

12         Q.    And some of the mathematical simplification that 

13    were used in 1990 used some of the assumptions that were 

14    used in the 1980 census adjustment model by Ericksen, isn't 

15    that correct?

16         A.    Well, the environment was very different, of 

17    course, in 90 than in 80.

18         Q.    But they used some of the same assumptions, 

19    didn't they, from the 1980 model in 1990; correct?

20         A.    Applied similar assumptions in a very different 

21    environment, yes.

22         Q.    So the answer is yes?

23         A.    Well, the answer is what I just gave.

24         Q.    Which is yes or no?  

25               THE COURT:  Yes.  
                                                              714

 1               MR. MILLET:  I don't have Mr. Zimroth's excuse, 

 2    your Honor, mine are both working.  

 3               MR. ZIMROTH:  That is not an excuse, Tom.  

 4               THE COURT:  It wasn't working then.  

 5               MR. MILLET:  Sometimes they misfire.  I 

 6    apologize.  

 7               (Pause) 

 8               MR. MILLET:  Your Honor, may I approach?  

 9               THE COURT:  Yes.  

10    BY MR. MILLET:

11         Q.    Dr. Wolter, I am showing you what has been marked 

12    for identification purposes as Defendant's Exhibit 67.  

13               MR. MILLET:  Your Honor will find excerpts that I 

14    am going to be referring to under tab 5 and 6, your Honor.          

15               (Pause) 

16    BY MR. MILLET:

17         Q.    Dr. Wolter, take a look at exhibit Defendant's 

18    Exhibit 67.  

19               THE COURT:  Exhibit 67?  

20               MR. MILLET:  The markings are different in your 

21    copy.  It has been marked as Exhibit 67, but the same items 

22    are included in your notebook under tabs 5 and 6.

23         Q.    These are copies of an article written by 

24    Freedman and Navidi, as well as a copy of the comment on 

25    that article which you wrote, are they not, Dr. Wolter?
                                                              715

 1         A.    Yes, yes, it is.

 2         Q.    And the subject of the main article by Freedman 

 3    and Navidi is the Ericksen/Kadane adjustment model for 1980, 

 4    is that correct?

 5         A.    Yes, 1980, not 1990, but 1980.

 6         Q.    Did I say 90?  

 7               THE COURT:  No, you said 80.  He said 90. 

 8               THE WITNESS:  Yes.  

 9               MR. MILLET:  He is exactly right.

10         Q.    If you would, take a look at your comment, Dr. 

11    Wolter, on page 24.  

12               MR. ZIMROTH:  Excuse me.  Our copy is missing a 

13    page, Mr. Millet.  Do you have another copy?  

14               MR. MILLET:  Sure.

15               (Pause)

16               THE WITNESS:  I seem to be missing every other 

17    page.  I don't seem to have 24, 26, 28.  There is an 

18    undercount in this book.  

19               MR. MILLET:  We will see if we can adjust it, Dr. 

20    Wolter.  

21               THE COURT:  Let's smooth it.  

22               Maybe this would be a good point to take a 

23    mid-afternoon stretch.  You the

24               (Recess) 

25    BY MR. MILLET:
                                                              716

 1         Q.    Dr. Wolter, if you would, please, take a look at 

 2    what has been marked as Defendant's Exhibit 67.

 3         A.    Okay.

 4         Q.    Those include an article by Dr. Freedman and Mr. 

 5    Navidi as well as a comment by you on the Freedman article, 

 6    is that correct?

 7         A.    That's correct.

 8         Q.    And the subject matter is the Ericksen/Kadane 

 9    adjustment model for 1980, isn't that right?

10         A.    That's correct.

11         Q.    Could you take a look at your comment, Dr. 

12    Wolter, referring to page 24, and I am going to specifically 

13    refer you to the left-hand column, third paragraph, the last 

14    sentence, which reads:

15               "At the margin, however, there will always be 

16    disagreements between statisticians about the acceptability 

17    or usefulness of any given model in any specific 

18    application." 

19               Do you see that reference, Dr. Wolter?

20         A.    I sure do.

21         Q.    Do you still agree with that, Dr. Wolter?

22         A.    I think so.

23         Q.    I would like you to take a look at the Freedman 

24    and Navidi article, particularly page 5, which we also have, 

25    your Honor, again produced on our chart on the easel there, 
                                                              717

 1    and I want to direct your attention specifically to the 

 2    numbered paragraphs on the right-hand column at the top, 

 3    numbered 1 through 7. 

 4               Do you see those?

 5         A.    Page 5?

 6         Q.    Page 5 of the Freedman and Navidi article.

 7         A.    Yes.

 8         Q.    Those outline the assumptions of the 

 9    Ericksen/Kadane 1980 adjustment model, do they not?

10         A.    Yes, sir.

11         Q.    And you agree that those assumptions failed in 

12    1980 to the extent that the model was not usable for 

13    adjustment purposes, right?

14         A.    Yes, sir.  There was bad data in 1980.

15         Q.    But you also agreed that these assumptions failed 

16    and made the model nonusable, correct?

17         A.    Right.  It was bad data and the model didn't fit 

18    the bad data.

19         Q.    Okay.  Fair enough.

20         A.    So we gave it up.

21         Q.    Take a look at the assumption at paragraph number 

22    1 of the Freedman article.

23               Wasn't an assumption like number 1 used in the 

24    1990 PES?  

25               MR. SOLOMON:  Your Honor, my only objection is 
                                                              718

 1    the same as the one I made at the deposition.  Questions 

 2    about whether an assumption is like another assumption or an 

 3    equation is like or similar to another equation is, I think, 

 4    like asking whether E equals MC squared is the same as E 

 5    equals MCQ, and my guess is they are very different.  I 

 6    think as a statistician and a mathematician we should be 

 7    more precise.  

 8               THE COURT:  Get your act together, that is the 

 9    burden of the objection.  

10               MR. MILLET:  Your Honor, I would also say that 

11    Dr. Wolter had no problem answering that question at the 

12    deposition even after Mr. Solomon's objection, and I would 

13    like to restate it and see if he can answer it today.  

14               THE COURT:  Yes.  If the witness is comfortable 

15    with it, he's the important one.

16         Q.    My question again, Dr. Wolter, wasn't an 

17    assumption like assumption number 1 used in 1990?

18         A.    Well, it's something like, yes.  I think the 

19    difference is there is a completely different PES design, 

20    there is a different environment, and this little subscript 

21    "i" that you see attached to all those Greek symbols there 

22    refers to area or post-strata.  That is completely different 

23    in 1990 than it was in 1980.

24               So when you say something like, I don't know what 

25    that means exactly.  That's not a technical term, but 
                                                              719

 1    setting that aside, this, this equation is something like 

 2    what's going on in 1990, but with all those qualifications.

 3         Q.    Number 2 is a linear equation with variables, is 

 4    it not?

 5         A.    Yes, it is, that's correct.

 6         Q.    Wasn't there a linear equation of this type used 

 7    in 1990, but with different variables?

 8         A.    Of this type, but with different variables, 

 9    that's correct.

10         Q.    Assumption number 3 was used in 1990, wasn't it?

11         A.    Again, something like it, yes.

12         Q.    You are not sure whether it is the same thing?

13         A.    As I was saying before, there is a completely 

14    different PES system in 1990 than there was in 1980, and 

15    once again, I would especially call everyone's attention to 

16    the "i" subscript.  That, again, signifies the post-stratum 

17    and they were very different in 1990 than in 1980.

18         Q.    Didn't you testify in your deposition that 

19    assumption 3 was used in 1990, and I'm talking about the 

20    second deposition that occurred in April of this year?

21         A.    If you asked me, I'm, I'm sure that I did, 

22    something like this assumption is used in 1990.

23         Q.    Now look at assumption number 4, that is sampling 

24    variability, isn't it?

25         A.    Number 4?
                                                              720

 1         Q.    Yes.

 2         A.    Yes, sir.

 3               Let's call it random error.

 4         Q.    Okay.

 5         A.    Sampling variability being one of the major parts 

 6    of that, yes.

 7         Q.    And that random error was used in 1990, was it 

 8    not?

 9         A.    Yes, sir.

10         Q.    Look at assumption 5.  An equation like 

11    assumption 5 was used in the smoothing model, wasn't it?

12         A.    In 1990?

13         Q.    Correct.

14         A.    Yes, something like that, yes.

15         Q.    And Freedman/Navidi's assumption 6 says the 

16    epsilon are independent, isn't that right?

17         A.    Yes, sir.

18         Q.    And that assumption was used in 1990, isn't that 

19    correct?

20         A.    Yes, I think that's what the Census Bureau did.

21         Q.    And assumption 7 talks about normal distribution.  

22    That was used in the smoothing model in 1990, wasn't it?

23         A.    Was used in the smoothing model?

24         Q.    Wasn't assumption 7 used in 1990?

25         A.    Offhand I can't think of where that assumption 
                                                              721

 1    was used in 1990.

 2               In some respects that assumption is used in part 

 3    in some of these intervals that we looked at earlier today, 

 4    but certainly in developing the smooth estimates, in 

 5    developing the corrected census counts, 7 doesn't figure.

 6         Q.    Seven doesn't figure in 1990?

 7         A.    That's not what I said.

 8               I said in calculating the smoothed estimates and 

 9    in calculating the corrected census figures, 7 doesn't 

10    figure.

11         Q.    Didn't you testify in your deposition in April 

12    of this year that the smoothing model encapsulated 

13    assumption 7?

14         A.    I doubt it.  These kinds of models don't, for 

15    many purposes, anyway, don't require us to make assumption 

16    7.  For some purposes we make an assumption like 7, but 

17    that's not always required, no.

18         Q.    Didn't you testify in your deposition that, "the 

19    smoothing model encapsulates assumptions 5, 6 and 7 or 5 and 

20    6 and part of 7, although, give me a minute to review what 

21    in the world the notation is here," looking at page 797?  

22               Do you have that in front of you? 

23               Do you have volume five of your deposition in 

24    front of you?

25         A.    I'm uncertain.  Do I?
                                                              722

 1         Q.    I will give it to you.

 2               (Handing to the witness)

 3               (Pause)

 4               Yes, here we go.

 5               Why don't you just take a look at page 797 and 

 6    tell me if that refreshes your recollection as to whether 

 7    you testified that assumption 7 was used in 1990.

 8         A.    Page 797?

 9         Q.    Page 797 at the bottom where it reads at line 23, 

10    "it encapsulates 5, 6 and 7 or 5 and 7 and part of 6, 

11    although --  give me a minute to review what in the world 

12    the notation is here.

13               "Q.    Please.

14               "A.    Well, having taken a little bit more of a 

15    look at it, I would say to you again that the third line 

16    from the bottom, page 7, is really very much the same as 

17    equation 5, part of 6 and 7 in Freedman." 

18               Does that refresh your recollection as to whether 

19    you testified that assumption 7 was used in 1990? 

20               (Continued on the next page) 

21    

22    

23    

24    

25    
                                                              723

 1               MR. SOLOMON:  Your Honor, the witness also makes 

 2    reference to assumption 7, 790, but perhaps his attention 

 3    could be directed there as well.  

 4               MR. MILLET:  You are free to do so, Mr. Solomon.  
                                                                   

 5    Right now I'd like to stay where we are.

 6               THE COURT:  Well, if he needs it to understand 

 7    what we've just read --

 8               MR. SOLOMON:  Thank you.  Take at look at 7.

 9               THE WITNESS:  Page 790 too?

10               THE COURT:  790 also.

11         A.    Well, on 790, I would direct your attention to 

12    line 11, where I stated and I quote:  

13               "Well, 7 is neither here nor there."

14         Q.    And I'm not asking you whether its here nor 

15    there.  I'm asking you whether it was used in 1990, and 

16    didn't you testify that it was?  Yes or no, Dr. Wolter?

17         A.    I think I testified at my deposition and tried to 

18    convey the information to you then that I'm trying to convey 

19    to you now.

20         Q.    So --

21         A.    And the information I'm trying to convey to you 

22    now is that for certain of these interval confidence 

23    statements that we were looking at earlier, we may want to 

24    make an assumption like 7 or something close to 7.

25               But for purposes -- for most other purposes, 7 
                                                              724

 1    wasn't used in calculating the corrected census data.

 2         Q.    So is page 798 wrong?

 3         A.    No.  I was -- I just said that what I was 

 4    conveying to you in the deposition is the same as what I'm 

 5    trying to tell you now.

 6         Q.    So it's your testimony that 7 was or was not used 

 7    in 1990?

 8         A.    I'm saying that it is used in some of the 

 9    Bureau's documents for some purposes, but it's certainly not 

10    used everywhere.

11         Q.    I didn't ask you if it was used everywhere.

12         A.    And in particular, it isn't used in the 

13    calculation of the adjusted census counts.

14         Q.    Take a look at assumption 6.  That's describing 

15    error terms, is that correct?  We are back on --

16         A.    Assumption 6, yes, sir.

17         Q.    And that indicates that the assumption is that 

18    the error terms are independent, isn't that right?

19         A.    Yes, sir.

20         Q.    If in 1990 the error terms were not independent, 

21    wouldn't that mean that the estimated variances were too 

22    small?

23         A.    (Pause) 

24               I don't know.  That's very hard to sit here and 

25    speculate about.  Could be.
                                                              725

 1         Q.    It could be?

 2         A.    Could be, yes.

 3         Q.    If the variances are smooth -- 

 4         A.    Wouldn't expect them to be very correlated, 

 5    wouldn't expect them to distort estimated variances very 

 6    much though.

 7         Q.    But wouldn't you think that's a real possibility 

 8    that the variances would be too small if the error terms 

 9    were not independent?

10         A.    Well, everything taken in context, my senses are 

11    the variances tend to be a little bit too large at times.

12         Q.    That's not my question.

13         A.    For this one narrow issue.  Could be too small, 

14    but taken in broader context, on balance, I don't see a 

15    problem with variance estimation.

16         Q.    I'm asking you if the error terms were 

17    independent, not in any other context.  If the error terms 

18    are independent, couldn't that mean that the estimated 

19    variances were too small?

20         A.    Could mean that.  We'd have to look at it a lot 

21    more carefully to know for sure.

22         Q.    Thank you.  

23               If the variances of the smooth adjustment factors 

24    were doubled, wouldn't that be of concern to you?

25         A.    If the variances of the smooth adjustment factors 
                                                              726

 1    were doubled.

 2         Q.    Right.

 3         A.    Well, all other things being equal, we want the 

 4    variances to be as small as possible.

 5         Q.    And if they were doubled, that would be of 

 6    concern, isn't that correct?

 7         A.    I would be more concerned than if they are what 

 8    they are.

 9         Q.    If you were presented with evidence that the 

10    variances should be doubled, isn't it true that you'd want 

11    to re-examine your recommendation to the Secretary that the 

12    1990 Decennial Census should be adjusted?

13         A.    If the variances were much larger than they 

14    are --

15         Q.    No, I'm asking you if they were doubled.

16         A.    Okay.  If they were doubled -- 

17         Q.    If you were presented with evidence that the 

18    variances should be doubled --

19         A.    Sir? 

20         Q.    -- isn't it true that you would want to reexamine 

21    your recommendation to the Secretary that the 1990 Decennial 

22    Census should be adjusted?

23         A.    I would think so.  I would think we would want to 

24    very carefully weigh in that new evidence with all the other 

25    evidence, just like the panel did in the spring and summer 
                                                              727

 1    of 1991.  

 2               MR. MILLET:  Thank you, Dr. Wolter.

 3               Your Honor, I have no further questions.

 4               THE COURT:  Thank you.  Any redirect?  

 5               MR. SOLOMON:  Very briefly.

 6               THE COURT:  Sure.  I have some swamp land in 

 7    Florida.

 8    REDIRECT EXAMINATION 
                           

 9    BY MR. SOLOMON:             

10         Q.    Dr. Wolter, if you have in front of you tab 22 of 

11    the notebook that you and I were working through, project 

12    P --

13         A.    Could I clean up my work area here?

14         Q.    All of their stuff, just toss it in the corner 

15    there.  

16               MR. MILLET:  I'm going to object to that.  

17               (Laughter)

18         Q.    In one big heap.

19               (Pause)

20         A.    Tab 22?

21         Q.    Please.  Very briefly.  Page 18.

22               Let us suppose that the B bias number for the 

23    nonminority aggregation here on table 18 contains a 

24    typographical error; let us suppose that the bias, instead 

25    of being 1.49 was .49, or something like that.  Would that 
                                                              728

 1    change any of the opinions that you offered to the Judge 

 2    concerning the evidence that is supplied by this table 

 3    concerning the differential undercount?

 4         A.    Certainly not.  That strengthens our knowledge of 

 5    the nonminority undercount; and, in any event, this is the 

 6    differential undercount, this difference between 5.6 and 

 7    1.1.  That is the striking and persistent feature of census- 

 8    taking in the United States, and if the 1.49 were a 

 9    typographical error, that wouldn't change that differential 

10    undercount.

11         Q.    You and Mr. Millet talked about project P 1, and 

12    he asked you about the mover alternative.  And in my view, 

13    he wouldn't let you answer the question.

14               I want to show you P 1, if I may.  This is 

15    Plaintiff's Exhibit 540 A E S.  May I give this to the 

16    witness, please?

17               THE COURT:  Sure.  

18               MR. SOLOMON:  Okay.

19         Q.    Under the summary in discussing the mover 

20    alternative, do you see that reference there?

21         A.    No. 3, paragraph 3?

22         Q.    Yes.

23         A.    Yes, sir.

24         Q.    And I think when you started to answer Mr. 

25    Millet, I think you said that the Bureau had rejected that 
                                                              729

 1    alternative as unreasonable.  Do you see there the Bureau is 

 2    calling that alternative an extreme one?

 3         A.    Yes, sir.

 4         Q.    Okay.  Is it extreme?

 5         A.    Well, the -- let's explain what a mover is in the 

 6    first place.  Okay?

 7               A mover is the Bureau's jargon in referring to a 

 8    PES individual that lived somewhere else on census day.

 9               Now, remember, the PES interview takes place in 

10    July.  The census took place back in April, and people move 

11    in between.  And that's what the Bureau calls movers.

12               I also stated this morning that mobile people 

13    tend to be undercounted to a higher extent than nonmobile 

14    people.  This is absolutely striking.  It's fundamental in 

15    census-taking.  People who are on the move have a higher 

16    propensity to be missed.

17               Now then, in the mover alternative imputation, as 

18    it's called here, what the Bureau did is it said:  All 

19    right, if data is missing for movers, let us, in this 

20    alternative, set aside the fact that we know they're a 

21    mover.  And, instead, let's hypothesis that they were a 

22    nonmover and that we didn't attempt any mention for them.  

23    And let us estimate their enumeration status as if they were 

24    nonmovers.

25               The effect of that is very simple:  It attributes 
                                                              730

 1    the -- it attributes to these movers now a lower match rate 

 2    -- excuse me.  Did I say that right?  It attributes to them 

 3    the higher match rate of nonmovers.  

 4               In other words, it says that now these movers are 

 5    nonmovers, and they match at a higher rate to the census.  

 6    Their undercount is less, and that's why in the table that 

 7    Mr. Millet showed us before, the undercount was lower than 

 8    all of the reasonable alternatives.  And, as I said before, 

 9    this is -- this is truly extreme.  Everyone at the Census 

10    Bureau knows that mobile people have a higher propensity to 

11    be mixed in the census.  

12               MR. MILLET:  Objection, and move to strike that.  

13    Everyone in the Census Bureau knows.  There's no foundation 

14    for that.

15               THE COURT:  All right.  It is stricken.

16               THE WITNESS:  Fair enough.  

17    BY MR. SOLOMON:

18         Q.    Was it generally understood at the Census Bureau 

19    during the time that you were there that the assumption that 

20    Mr. Millet -- the only assumption that Mr. Millet showed you 

21    was an extreme and unreasonable one?

22         A.    Well, yes, that was the only one we looked at in 

23    P 1.

24         Q.    Okay.  Was there any respect in which you and 

25    Professor Ericksen and Tukey and Estrada in your report 
                                                              731

 1    intended to depart from the Census Bureau data in analyzing 

 2    the PES?

 3         A.    I don't believe there was any intent on my part.  

 4    I don't believe there was any intent on their parts to 

 5    deviate from Census Bureau data.  The Census Bureau is the 

 6    fountain of all this data.  And we certainly meant to 

 7    utilize to the maximum extent feasible their data, their 

 8    findings, their experiences.

 9         Q.    The last point.  Mr. Millet talked with you about 

10    your comment on the Freedman-Navidi article, and he read you 

11    a piece and talked to you about some questions for the 

12    truth.  I don't really know what we want to do with that.  

13               I would, however, like you to look at page 27 of 

14    that comment.  Do you have that?  I can give you a copy, if 

15    you want.

16         A.    I have it.

17         Q.    Item No. 14.  Where you are summing up or 

18    concluding, please read to the Court what you say there.

19         A.    Item No. 14, and I quote:

20               "So where does this leave us in terms of the 

21    future?  It seems to me that the models of the kind 

22    discussed here may indeed be useful for adjusting the 1990 

23    Census.  It also seems to me that the critical world 

24    involves the direct undercount estimators by themselves.  If 

25    the undercount programs for 1990 can be conducted with 
                                                              732

 1    little nonsampling error, particularly biased relative to 

 2    the presumed size of the undercount, then there is a good 

 3    likelihood that models of the kind discussed here will be 

 4    acceptable and adjusted population counts derived from such 

 5    models will be closer to the truth than the unadjusted 

 6    census counts.

 7               "In addition to this critical work, we must 

 8    carefully check the remaining assumptions and modify the 

 9    methodology where necessary and use a unit of analysis that 

10    is comparable to the areas received in adjustments."

11         Q.    As a result of the analysis that you performed 

12    and the Bureau performed, did you in fact conclude that the 

13    models that were used enabled you to have counts, adjusted 

14    counts, that were closer to the truth than the unadjusted 

15    census counts?

16         A.    Yes, both in absolute terms and distribution.  

17               MR. SOLOMON:  Thank you very much.

18               THE COURT:  Thank you.  

19               MR. MILLET:  Very short recross, your Honor.  

20    RECROSS-EXAMINATION 
                          

21    BY MR. MILLET:

22         Q.    Dr. Wolter, you talked about movers.  Isn't it 

23    true that mobility is not a variable in the 1990 production 

24    regression model?

25         A.    Is not?
                                                              733

 1         Q.    Is not -- mobility is not a variable in the 1990 

 2    production regression model.

 3         A.    I think that's true, that's correct.

 4         Q.    Now, Dr. Wolter, isn't it true that 2.11 is not 

 5    the same as 2.06?

 6               MR. SOLOMON:  I'm prepared to stipulate to that, 

 7    also.

 8               THE COURT:  I want proof.

 9         Q.    Will you also stipulate that 2.11 is not the same 

10    as 2.16, 1.65 or 2.07?  

11               MR. SOLOMON:  I have no more -- let him answer. 

12         Q.    Those are different answers, aren't they, Dr. 

13    Wolter?

14               THE COURT:  Yes.

15         A.    Yes, sir, those are different numbers.  

16               MR. MILLET:  Thank you, your Honor, no more 

17    questions.  

18               MR. SOLOMON:  He raised another matter.  

19    FURTHER REDIRECT EXAMINATION
                                  

20    BY MR. SOLOMON:

21         Q.    In the regression model, mobility wasn't taken 

22    into account directly.  How was mobility taken into account 

23    in the regression model?

24         A.    Well, mobility is well-correlated with age, it's 

25    well-correlated with type of place, it's well-correlated 
                                                              734

 1    with some of the other information that was included in this 

 2    smoothing process.  

 3               MR. SOLOMON:  Thank you.

 4               THE COURT:  Thank you.  I believe we have 

 5    completed.

 6               Pleasure to listen to you.

 7               THE WITNESS:  It was a pleasure.  Thank you.

 8               THE COURT:  See you in ten years.

 9               (Witness excused)
                                  

10               THE COURT:  What do you want to do now?  

11               MR. RIFKIND:  We'd like to proceed with our next 

12    witness, if we may, your Honor.  

13               MR. MILLET:  May we have maybe two minutes to 

14    clear off and get ready for the next witness?

15               THE COURT:  Fine.  Two minutes?  

16               MR. RIFKIND:  Two-minute break.

17               (Recess)

18               (In open court) 

19               MR. RIFKIND:  Your Honor, the next witness will 

20    be examined by my colleague, Mr. Sherman.  

21               MR. SHERMAN:  Good afternoon, your Honor.  

22    Plaintiffs call John E. Rolph.  

23               THE COURT: Good afternoon.

24    JOHN E. ROLPH, 
                     

25         called as a witness by the plaintiffs, having
                                                              735

 1         been duly sworn, testified as follows:

 2               THE COURT:  Mr. Sherman.  

 3               MR. SHERMAN:  Thank you, your Honor.  

 4    DIRECT EXAMINATION 

 5    BY MR. SHERMAN:

 6         Q.    Dr. Rolph, what is your occupation?

 7         A.    I'm a statistician on the research staff of the 

 8    Rand Corporation.

 9         Q.    How long have you been a statistician?

10         A.    I believe I had my first job as a statistician in 

11    the summer of 1962.  I've guess I've been a card-carrying 

12    member of the profession since 1966 when I received my Ph.D.

13         Q.    What is your area of specialization?

14         A.    Methodologically, I specialize in empirical Bayes 

15    methods.  In application, I specialize in the applications 

16    of statistics to public policy questions.

17         Q.    What are empirical Bayes methods, Dr. Rolph?

18         A.    Empirical Bayes methods are statistical 

19    estimation methods that are set up to improve the accuracy 

20    of a set of estimators by sharing information between them.

21         Q.    Is it a new methodology?

22         A.    No.  Its roots probably go back to the 1800s.  

23    It's been in use in education and psychology perhaps since 

24    the '20s or so, and it's been in more widespread use for, I 

25    would say, maybe 30 years, something like that.
                                                              736

 1         Q.    Were empirical Bayes methods used by the Census 

 2    Bureau in the 1990 Census?

 3         A.    Yes, they were.

 4         Q.    How were they used?

 5         A.    In the 1990 Census, the Census Bureau used the 

 6    term "smoothing" which is basically a synonym for empirical 

 7    Bayes methods, and they were used to improve the accuracy of 

 8    the estimate of the raw adjustment factors.

 9         Q.    Then the smoothing procedures used by the Bureau 

10    fall within your area of specialization?

11         A.    Yes, I would say that they do.

12         Q.    Dr. Rolph, I'd like to call your attention to 

13    Plaintiff's Exhibit 56, which should be in the binders -- 

14               (Pause)

15         A.    Okay.  I have it now.  

16               MR. SHERMAN:  We'll just wait for the Judge to 

17    get it.

18               THE COURT:  All right. 

19         Q.    Do you recognize PX 56, Dr. Rolph?

20         A.    Yes.  That's a copy of my curriculum vitae.

21         Q.    Is it an up-to-date version of that?

22         A.    Yes.  It's April 1992, so I believe I updated it 

23    last month.  

24               MR. SHERMAN:  Your Honor, at this time I move 

25    PX 56 into evidence.  
                                                              737

 1               MR. BARON:  No objection.

 2               THE COURT:  56 is admitted.  

 3               (Exhibit PX 56 for identification was received in 

 4    evidence)

 5    BY MR. SHERMAN:

 6         Q.    Dr. Rolph, would you describe your educational 

 7    background, please.

 8               THE COURT:  You can do it rather briefly.  I have 

 9    this in front of me and I can read.  Why not highlight 

10    whatever you are most proud of.

11         A.    I received a Bachelor's Degree in statistics and 

12    a Ph.D. in statistics at the University of California 

13    Berkeley.

14         Q.    All right.  I'll try to help with the highlights.  

15    Can you tell us what the Rand Corporation is, Dr. Rolph?

16         A.    Yes.  It's a research institution in Southern 

17    California which has been in existence since about World War 

18    II; does research on public policy issues in the national 

19    interest.

20         Q.    What sort of work do you personally do at Rand?

21         A.    I, in addition to working on statistical 

22    methodology, do work in a variety of different application 

23    areas.  

24               I'm currently heavily involved in medical 

25    malpractice; that is, not malpracticing medicine, but 
                                                              738

 1    studying it; but have done a fair bit of work over the years 

 2    in the health work, particularly looking at health insurance 

 3    questions, in addition to medical school, performance in 

 4    medical school, automobile accident compensation, estimated 

 5    fire alarm incidents, and frequencies, and a variety of 

 6    other public policy issues.

 7         Q.    Do you ever use census data in your work?

 8         A.    Yes, I do, from time to time.

 9         Q.    Do you use empirical Bayes methodology in your 

10    work?

11         A.    Yes, I do.

12         Q.    Have you received any honors in your professional 

13    career?

14         A.    Yes.  In 1988, I was elected as a Fellow of the 

15    American Statistical Association, and in 1989, a Fellow of 

16    the Institute of Mathematical Statistics.

17         Q.    What is the significance of being a Fellow in 

18    those organizations?

19         A.    These are the two major statistical organizations 

20    in the United States, and they honor essentially 

21    statisticians for making significant contributions to the 

22    field, and I was fortunate in being so honored.

23         Q.    What were your contributions to the field?

24         A.    The citation for the Fellowship read something 

25    along the lines for research in empirical Bayes methods, for 
                                                              739

 1    creating a research group of statisticians working in public 

 2    policy problems, and for service of the profession in the 

 3    form of editorial service and serving on national 

 4    committees.

 5         Q.    Have you done consulting outside of Rand that 

 6    would involve issues of relevance to this proceeding?

 7         A.    Yes.

 8         Q.    Would you give us a few of those, the highlights?

 9         A.    The highlights.  Yes.  

10               Of relevance to using census data for about the 

11    last ten years, I've been consultant to the Los Angeles 

12    Superior Court.  My role has been in helping them devise a 

13    method for selecting jurors and then assigning them to the 

14    various court locations in California.

15         Q.    All right.  Have you published any articles or 

16    books of relevance to this case?

17         A.    Yes.  It's as outlined in my CV.  I won't read 

18    it.

19               There are -- there's a statistics textbook, a 

20    couple of chapters in different books, and then a number of 

21    articles and reports as part of my work at the Rand 

22    Corporation.

23         Q.    Are there any in the list that are of particular 

24    relevance to this proceeding?

25         A.    Yes.  A number of them use empirical Bayes 
                                                              740

 1    methods.  

 2               Let me start with the first one.  This was -- 

 3    this came out of some work here in New York City in which 

 4    Rand was asked to help the fire department devise better 

 5    rules for dispatching fire engines to fires.  And one of the 

 6    issues was to try to estimate the probability that a 

 7    particular fire alarm signal signals a serious fire as 

 8    opposed to a less serious fire, a rubbish fire.  

 9               We essentially divided up the borough of the 

10    Bronx into neighborhoods, if you will, and used empirical 

11    Bayes methods, if you will, to smooth the estimates in much 

12    the same way census data is smoothed.

13               In addition, -- this is going to sound like this 

14    is all I use -- I have used empirical Bayes methods in my 

15    study of medical malpractice, in some of the statistical 

16    work I've done on analyzing different health insurance 

17    plans, and the effect it makes on health care spending, on 

18    modeling criminal careers with an eye to identifying those 

19    criminals that are high-rate, serious offenders, and there 

20    are probably three or four others if we went through the 

21    list.

22         Q.    Dr. Rolph, with regard to the fire alarm study 

23    that you mentioned, was anything about that study ever used 

24    by the Census Bureau?

25         A.    Not by the Census Bureau directly, in terms of 
                                                              741

 1    the actual output of the study, but the methods that were -- 

 2    that we devised to estimate fire alarms were used in a later 

 3    paper by Roger Harriet and Bob Fay that appeared several 

 4    years later which was concerned with estimating the income 

 5    in the 39,000 revenue-sharing areas of this country; and, 

 6    basically, some of the methods in the fire alarm study we 

 7    used for their study.

 8         Q.    And Messrs. Fay and Harriet were at the Census 

 9    Bureau?

10         A.    Yes, both of them at the time were statisticians 

11    at the Census Bureau.

12         Q.    Have you been qualified as an expert in previous 

13    litigation, Dr. Rolph?

14         A.    Yes, I have.

15         Q.    Approximately how many times?

16         A.    I would say probably eight or ten times.

17         Q.    Okay.  What were some of the issues involved?

18         A.    Probably the most fun, the most recent case was 

19    son casino winnings, trying to determine whether the 

20    blackjack tables at a particular casinos were low due to 

21    chance alone or were, actually, there was some cause behind 

22    it.  

23               I've also done some work in several cases in jury 

24    representativeness.  This grew out of my work for the Los 

25    Angeles Superior Court in addressing whether or not a 
                                                              742

 1    particular jury system was representative of the area that 

 2    it served, and that was one of the uses of census data.

 3               Other areas include using regression in the 

 4    school finance case, and three or four other miscellaneous 

 5    items that escape me at the moment.  

 6               MR. SHERMAN:  Your Honor, at this time I would 

 7    offer Dr. Rolph as an expert in statistics and empirical 

 8    Bayes methodologies. 

 9               MR. BARON:  I have no objection.

10               THE COURT:  Dr. Rolph is qualified as an expert 

11    and may express his opinion.           

12    BY MR. SHERMAN:

13         Q.    Dr. Rolph, do you currently serve in any 

14    committees specifically related to the census?

15         A.    Yes, I do.  I am currently a member of the 

16    American Statistical Association Census Advisory Committee 

17    and have been since this past October, I believe.

18         Q.   What does that committee do?

19         A.   The Census Bureau has a number of committees that 

20    it advises in, one of them is from the American Statistical 

21    Association.  This committee meets twice a year with the 

22    senior managers of the Census Bureau, is consulted on 

23    various issues that they are concerned about, gives advice, 

24    listens to plans, reacts to it, that sort of thing.

25         Q.    In the past have you served on any committees 
                                                              743

 1    related to the census?

 2         A.    Yes.  During the 1980s, I served on the panel for 

 3    decennial census methodology.  This was a panel of the 

 4    National Academy of Sciences that was commissioned to advise 

 5    the Census Bureau on issues leading up to the 1990 Census -- 

 6    specifically, methodological issues.

 7         Q.    When was that panel formed?

 8         A.    My best recollection was the panel was formed in 

 9    1983, I believe, and had its first meeting in January of 

10    1984.  It, in fact, was two panels.  I think the second 

11    panel finally terminated in late 1988.

12         Q.    Okay.  For now, concentrating on the first panel. 

13               Did the Census Bureau request the advice of the 

14    National Academy panel on census matters?

15         A.    Yes, it did.

16         Q.    Do you know why?

17         A.    Yes.  My understanding is that the Census Bureau 

18    has always sought outside expert advice on issues before it.  

19    This particular panel was charged with giving advice on 

20    several issues.  One of them had to do with correcting the 

21    census for the undercount; and, in particular, the concern 

22    was, of course, about the differential undercount.

23         Q.    Why was the Bureau concerned with the possibility 

24    of correcting that count?

25         A.    Well, there has been -- I assume it has come out 
                                                              744

 1    already.  There's been persistent evidence for several 

 2    decades that in censuses in this country, the population is 

 3    not covered evenly; that is to say, different portions of 

 4    the population are counted well or less well.

 5               The particular timing here, of course, was 

 6    following the 1980 census, and there was a certain amount of 

 7    controversy about it then, and the Bureau was seeking expert 

 8    advice to the 1990 Census in taking the census as best they 

 9    could.

10         Q.    Did the National Academy panel discuss the 

11    problem of the differential count with the Bureau?

12         A.    Yes, we did.  It was one of several issues that 

13    were on the table.  We had read a number of papers from 

14    Census Bureau staff, had presentations from them, and the 

15    like.

16         Q.    How often did the National Academy panel meet?

17         A.    My best recollection is maybe four times a year 

18    for the first panel, something like that.

19         Q.    How long were the meetings?

20         A.    Typically -- they were in Washington, D.C., and 

21    typically they were two days, sometimes 2 1/2 days, 

22    occasionally a day and a half.

23         Q.    What would happen at the meetings?

24         A.    Generally, depending on exactly what stage of our 

25    work we were in, they consisted of, I would say, four 
                                                              745

 1    different activities.

 2               First, we would have presentations from Census 

 3    Bureau staff on issues that were concerning them.  Dr. 

 4    Wolter, for example, talked to us, Dr. Bailar, a number of 

 5    other people that had been involved in census matters.

 6               Secondly, we would get presentations from outside 

 7    experts on issues that the committee is deliberating on.  I 

 8    believe at our first meeting we had a presentation from 

 9    Professor Tukey.  I believe Dr. Ericksen, as I recall, gave 

10    us a presentation.

11               Third, we had discussions amongst ourselves, 

12    basically, trying to understand the issues that we were 

13    being asked to give advice on, trading opinions on them, 

14    trading information, and so forth. 

15               And last, but not least, one of the charges of 

16    this panel was actually to produce something for all this 

17    effort, and so we would work on preparing a report, even to 

18    the point of actually doing some writing at the meetings, 

19    talking about the recommendations, that kind of thing.

20         Q.    Dr. Rolph, you mentioned Drs. Ericksen and Tukey.  

21    The Court has already had occasion to meet Dr. Ericksen. 

22               Who's Dr. Tukey?

23         A.    Professor Tukey is -- I guess I would have to 

24    say, if not the most eminent statistician in the country, 

25    certainly one of them that is currently living.  He's been a 
                                                              746

 1    professor at Princeton for many years and with Bell 

 2    Telephone Laboratories for many years also.  He's done 

 3    pioneering work in a variety of areas including exploratory 

 4    data analysis, regression, and other areas.

 5         Q.    Dr. Rolph, if you would find PX 2 in your binder, 

 6    please.

 7         A.    Okay I have it.

 8         Q.    And I'd like to direct your attention to page 

 9    iii.  It's actually the second printed page in there.      

10         A.    Okay, I've got it.

11         Q.    Do you recognize the names on that page?

12         A.    Yes.  These are the names of the members of the 

13    first panel on decennial methodology which was chaired by 

14    John Pratt, who is a statistician at Harvard University.

15         Q.    What professions are these individuals drawing 

16    from?

17         A.    A number of them are statisticians.  There's also 

18    some representation from demography and at least one in 

19    economics.

20         Q.    Were these individuals prominent members of their 

21    field?

22         A.    Yes, I would say they are in my judgment.

23         Q.    Would it be fair to call this group blue-ribbon 

24    panel experts?

25         A.    That sounds fitting to me.
                                                              747

 1         Q.    Dr. Rolph, when the National Academy panel was 

 2    formed, did these individuals have a common view on the 

 3    issue of adjustment?

 4         A.    No, they did not.

 5         Q.    How do you know?

 6         A.    Well, for two reasons.  First of all, we 

 7    discussed the issue of the differential undercount in the 

 8    census and the possibility of correcting for it.  And just 

 9    from going around the table, there was clearly differences 

10    of opinion as to how possible or not possible that was with 

11    current data that was available.

12               Secondly, there were members of the panel that 

13    had testified on either side of the 1980 case, so that was, 

14    if you will, empirical evidence there was not a total 

15    meeting of the minds on the issue.

16         Q.    During the course of the National Academy of 

17    panel's existence, did the differences of opinion change?

18         A.    Yes, it did.  During the course of this 

19    particular panel's life, we started out, as I indicated, 

20    having a wide diversity of views.  As we had discussions 

21    among ourselves, presentations from both the Bureau staff as 

22    well as outside people, read some of the Bureau's papers and 

23    documents, memos and so forth, we eventually arrived to the 

24    place where we all unanimously recommended that the Bureau 

25    proceed with plans to adjust the 1990 Census and to carry 
                                                              748

 1    out an adjustment if it is so warranted.  

 2               MR. BARON:  Your Honor, Dr. Rolph can certainly 

 3    talk about what he discussed to anyone, but he can certainly 

 4    not talk about hearsay.

 5               THE COURT:  Not talk about?  

 6               MR. BARON:  What other individuals may or may not 

 7    have discussed en banc.

 8               THE COURT:  Well, he was talking with them.  I 

 9    think it is appropriate.  If it's a discussion between two 

10    other people, leave it out.

11               THE WITNESS:  Okay.  

12    BY MR. SHERMAN:

13         Q.    And wasn't your testimony just now that the 

14    recommendation was unanimous?

15         A.    Yes, it was.  There is in National Academy panels 

16    an opportunity for dissenting, and circulation of the report 

17    is put out, and that, in fact, happened in this case and 

18    nobody dissented.

19         Q.    Dr. Rolph, have you ever stated before today your 

20    view that the National Academy panel proceeded from a 

21    diversity of perspectives to its unanimous recommendation?

22         A.    Yes, I believe I have.

23         Q.    When did you do that, sir?

24         A.    I think I did it in writing in a letter I wrote 

25    to then director of the census, Dr. John Keane.  If my 
                                                              749

 1    memory serves me correctly, maybe in 1987 or so, in which I 

 2    described substantially the process that I described to you 

 3    just now.

 4         Q.    I draw your attention to PX 58 in your binder.  

 5    That's the letter to which you referred?

 6         A.    Yes, that is the letter, not a great copy, but 

 7    it's the letter.

 8         Q.    Dr. Rolph, can you either pull the microphone 

 9    toward you or --

10         A.    Thank you.  Is this a little better?  Okay?

11               (Discussion off the record) 

12               MR. SHERMAN:  Your Honor, at this time I move 

13    PX 58 into evidence.  

14               MR. BARON:  No objection.

15               THE COURT:  58 is admitted.  

16               (Exhibit PX 58 for identification was received in 

17    evidence)

18    BY MR. SHERMAN:

19         Q.    Dr. Rolph, does this letter to Dr. Keane also 

20    describe how the National Academy panel came to its 

21    unanimous conclusion?

22         A.    Yes, it does.

23         Q.    Would you tell us exactly how it does so?

24         A.    Yes.  As I state in the letter, we started out 

25    having a wide range of opinions; and two things happened: 
                                                              750

 1    One, we got more information; and, two, as the Census Bureau 

 2    carried out its research program on the subject of 

 3    adjustment methodology, the panel became more and more 

 4    persuaded that they were solving the problems or were 

 5    actually addressing the issues that needed to be addressed 

 6    to produce an accurate adjustment.  

 7               So by the time I wrote this letter, which is in 

 8    April of 1987, the panel had reached a consensus that, 

 9    indeed, the Bureau should plan for an adjustment.  And if I 

10    could just read the last two sentences of the first 

11    paragraph, I think that sums it up.  They read:  

12               "The panel made this recommendation as a result 

13    of our critical appraisal of the cost-effectiveness of the 

14    1988 Coverage Improvement Programs of the substantial 

15    differential undercount that no adjustment decision applies 

16    and of the problems that must be successfully addressed in 

17    order to carry out a reasonable accurate adjustment of 

18    counts.  At the outset I would not have predicted this 

19    outcome."

20         Q.    Thank you.  

21               Dr. Rolph, did the National Academy panel work 

22    with the Bureau and outside experts in their consultation 

23    with the Bureau?

24         A.    Yes, we did.

25         Q.    And in so doing, did the National Academy panel, 
                                                              751

 1    the Bureau and other outside experts consider the issues 

 2    that had been raised with regard to adjustment in the 1980 

 3    census?

 4         A.    Yes, we considered, I believe, most of those 

 5    issues.

 6         Q.    Would you discuss some of the issues that were 

 7    considered.

 8         A.    Well, from memory, we certainly covered issues 

 9    like issues of matching, issues of imputation, issues of 

10    doing a capture-recapture model, the dual system estimate 

11    that was used in the 1990 adjustment process.  

12               We had discussed issues of what was commonly 

13    known to become smoothing, we discussed issues of so-called 

14    homogeneity of the poststrata for the capture-recapture 

15    model, and there are probably two or three that I'm leaving 

16    out from my memory.

17         Q.    Did the National Academy panel consider the issue 

18    of correlation bias?

19         A.    Yes.  That was part of our -- at least I lumped 

20    that together really with the issue of homogeneity of the 

21    poststrata.

22         Q.    Did the National Academy panel consider the issue 

23    of small area estimation?

24         A.    Yes, I believe -- I think we had a chapter, at 

25    least an appendix on it.  Basically, we discussed it at some 
                                                              752

 1    length.

 2         Q.    Did you consider the issue of survey nonresponse?

 3         A.    Yes, we did.

 4         Q.    And did you consider techniques to compare the 

 5    accuracy of the census to an adjustment?

 6         A.    Yes.  One of the issues that the panel was 

 7    charged with and did address was the notion of a yardstick 

 8    to measure the accuracy of the correction.  And the 

 9    technical term for that is a loss function, and we developed 

10    some recommendations about the type of loss functions that 

11    should be used in judging a correction.

12         Q.    And after considering all of those issues, did 

13    the National Academy panel make a former recommendation to 

14    the Bureau?

15         A.    Yes, we did.  We -- in, I believe it was, chapter 

16    7 of our report -- made a recommendation on the types of 

17    loss functions the Bureau should use.  I can summarize it I 

18    think fairly, fairly briefly.

19               First, we recommended that the Bureau use more 

20    than one loss function, that is, use several of them.

21               Secondly, we recommended that the Bureau consider 

22    in its loss functions the shares or proportions of the 

23    various geographical areas and how accurately they were 

24    estimated in addition to just the absolute numeric accuracy.

25               Third, we recommended that the Bureau, in 
                                                              753

 1    assessing the contributions in a given geographical area, 

 2    make sure that that was weighted proportionately to the 

 3    population in contrast to what we specifically recommend the 

 4    Bureau not do.  We recommended that they not simply count up 

 5    the political jurisdictions that were made better or worse 

 6    off by an adjustment.

 7         Q.    Dr. Rolph, were all the recommendations of the 

 8    first panel that you described embodied in the book called, 

 9    "The Bicentennial Census, New Directions for Methodology in 

10    1990"?

11         A.    Yes, they are.

12         A.    That -- I should add that was the final report of 

13    this panel.  And it was one of two reports that we issued.

14         Q.    And that's Plaintiff's Exhibit 2, correct?

15         A.    That's correct, it is.

16         Q.    And all those recommendations were reached 

17    unanimously?

18         A.    Yes, they were.

19         Q.    What are the National Academy panel's conclusions 

20    about the census, Dr. Rolph?

21         A.    I probably won't be able to reproduce the wording 

22    exactly, but, basically, we described the census over the 

23    past 200 years as a process in which it is impossible to 

24    literally count every single person with perfect accuracy.  

25    And so the census, by its very nature, is an estimate of the 
                                                              754

 1    population.  

 2               Realizing that, we recommended that, No. 1, the 

 3    Census Bureau take as accurate a census as was practically 

 4    possible, and, No. 2, given that there was inevitably going 

 5    to be a differential undercount, that it plan to correct for 

 6    it using appropriate statistical methods.

 7         Q.    Did the National Academy panel say anything 

 8    relevant to the notion that the census is a 200-year 

 9    tradition of actually counting people?

10         A.    Yes.  I think the substance of that, as best I 

11    can recall, was along the lines of what I just said, namely, 

12    the Census Bureau has been trying to count people and doing 

13    a quite good job of it under the circumstances.  But, in 

14    point of fact, every record that you see coming out of a 

15    census does not correspond to that person being interviewed 

16    and being counted.  There are a number of variants on that.  

17    I'm sure you heard about them in this case.

18         Q.    And did the National Academy panel point out that 

19    the census is an estimate rather than a true count?

20         A.    Yes.  As I explained, we very much did.  

21               In particular, I think we commented explicitly on 

22    the issue that the Bureau was headed to address when they 

23    are looking to try to figure out whether a given housing 

24    unit is occupied or not, and after having repeatedly visited 

25    it, had not been able to locate somebody but had a good 
                                                              755

 1    reason to believe that somebody lived there, and impute the 

 2    household into that.  That is a statistical information 

 3    technique.  

 4         Q.    And you agreed with that?

 5         A.    Yes.  That's what the Census Bureau has been 

 6    doing for the last couple of censuses.

 7         Q.    Did the National Academy panel, in 1985, make a 

 8    recommendation on the issue of adjustment?

 9         A.    Yes, we did.

10         Q.    What was it?

11         A.    In 1985, we recommended that the Bureau plan for 

12    an adjustment and to carry out an adjustment if the data so 

13    warranted it.

14         Q.    And, again, that was an unanimous recommendation?

15         A.    That was the unanimous recommendation concerning 

16    the 1990 Census.

17         Q.    Dr. Rolph, what happened to the National Academy 

18    panel on decennial census methodology after you issued the 

19    1985 volume which is PX 2?

20         A.    Well, that was the formal end of the first panel.  

21    Things had developed into a quite successful collaboration 

22    between the panel and the Census Bureau.  

23               The Census Bureau came back and wanted to have a 

24    second panel.  That second panel was constituted -- 

25    actually, what it was was the first maybe three-quarters of 
                                                              756

 1    the first panel.  Two or three people left, and two or three 

 2    new people came on, so starting, I'm going to guess, in 1986 

 3    or so, we had the beginning of a new panel, and the 

 4    objective of that new panel was to advise the Census Bureau 

 5    about the various dress rehearsals leading up to the 1990 

 6    Census.  

 7               And at the time it was even envisaged that we 

 8    would -- our life would extend up to and including the 1990 

 9    Census so that we could offer advice to the Bureau as part 

10    of their process in trying to correct the 1990 census.

11         Q.    And were you a member of the second National 

12    Academy panel as well?

13         A.    Yes, I was one of the hold-overs, as it were.

14         Q.    And did you say the second National Academy panel 

15    continued to work with the Bureau?

16         A.    Yes.  We worked actually extremely close with the 

17    Bureau during that second panel.

18         Q.    Did you ever work with outside experts?

19         A.    Yes.  From time to time, our appropriate outside 

20    experts in fact attended our meetings.  For example, I can 

21    recall one meeting in which, again, Dr. Ericksen was there, 

22    Dr. Tukey, and Professor Freedman was at one of our 

23    meetings.  There were a number of outside experts that came 

24    to our meetings.

25         Q.    Did the second National Academy panel review 
                                                              757

 1    papers on the PES produced by Bureau staff members?

 2         A.    Yes, we did.  Before each meeting, we typically 

 3    would get a binder, perhaps not as thick as this one, maybe 

 4    half the thickness, and I suppose if you were a fast reader 

 5    on the airplane, you could get it done before the meeting. 

 6               I'm sorry.  That was a facetious answer.  I'm 

 7    sorry.  

 8               Yes, we did read a lot of their papers.

 9               THE COURT:  Not very.

10               THE WITNESS:  Perhaps.

11               THE COURT:  Statistician's humor. 

12               MR. SHERMAN:  We've all gotten use to it, your 

13    Honor.  We are still trying to figure it out.

14         Q.    Did the second National Academy panel have 

15    briefings with the Bureau staff?

16         A.    Yes.  The formality for the second panel was very 

17    similar to that of the first, although it was much more a, 

18    if you will, roll-up-your-sleeves panel in the sense that 

19    the Bureau, at the time the second panel started, had a 

20    substantial research program under way to deal with the 

21    issues for adjusting the 1990 Census.

22               We worked quite closely with them in listening to 

23    what their solutions to the various issues were being 

24    addressed, making suggestions when the occasions warranted 

25    it, and, basically, it was one of these, I would call, post- 
                                                              758

 1    working relationships as opposed to a panel who took three 

 2    steps back and issued more formal proclamations, if you 

 3    will.

 4         Q.    What were some of the issues that you worked on 

 5    with the Census Bureau on the second-only panel?

 6         A.    The second panel, my recollection is there were 

 7    several issues.  The two that come immediately to mind were, 

 8    No. 1, set of issues surrounding the adjustment decisions, 

 9    specifically, the methodology to be used to carry out the 

10    adjustment, and, secondly, the criteria that should be used 

11    to make a judgment at the time of the 1990 Census whether or 

12    not to go ahead and report the adjusted numbers as opposed 

13    to report the unadjusted numbers.  That was one area.

14               The second area that comes to mind is we gave the 

15    Bureau advice on the experiments that they were running both 

16    as part of their dress rehearsals leading up to the 1990 

17    census as well as the experiments that were contemplated as 

18    part of the census itself.

19         Q.    So did the second panel give the Bureau advice 

20    and comments on the test censuses carried out in the late 

21    1980s?

22         A.    I believe we gave extensive comments on the 1986 

23    Los Angeles test.  That was the so-called TARO, which stands 

24    for test of adjustment related operations.  I believe by the 

25    time the 1988 dress rehearsal results were in, the panel had 
                                                              759

 1    since disbanded so that we did not comment on those.

 2         Q.    Would you briefly describe the second National 

 3    Academy panel's involvement in the TARO?

 4         A.    Yes.  The Bureau, as I indicated, was planning to 

 5    carry out -- to carry out an adjustment process as part of 

 6    the 1990 Census, including taking a post-enumeration survey 

 7    and the whole panoply of activities that are the subject of 

 8    this case.

 9               In 1986, they ran a test of adjustment-related 

10    operations; that is, they ran a census and they ran a PES; 

11    they put them together into dual system estimate and 

12    produced adjusted counts.  And it was that activity that we 

13    were advising the Bureau on and they reported to us on.

14         Q.    What was the panel's view of the results of the 

15    TARO operation?

16         A.    Well, we were quite positive about it, actually.  

17    In the letter you referred to earlier, to the director of 

18    the Census Bureau, Dr. Keane, we explicitly commented -- I 

19    beg your pardon.  That was a different letter to Dr. Keane.

20               We were quite positive on it, and in point of 

21    fact, we felt that things were going extremely well.  In 

22    fact, as well as could evenly be expected.

23         Q.    Did the National Academy panel issue a formal 

24    report following TARO?

25         A.    Yes, I think I got ahead of you.  
                                                              760

 1               We did write a letter to Dr. Keane in which we 

 2    commended the Bureau for the progress in the test of 

 3    adjustment related operations in 1986, and we also spoke 

 4    about the progress that the Bureau had made in addressing 

 5    the issues that needed to be addressed in order to carry out 

 6    successful adjustment in 1990.

 7               Indeed, I think probably the most impressive or 

 8    telling finding of that, as I recall, was a meeting which we 

 9    had in which Dr. Wolter and a number of his staff came and 

10    basically laid out in some detail how the 1986 test went, 

11    what the error rates were, and sort of what the numbers were 

12    that came out in terms of what the corrections should be for 

13    that particular test.

14         Q.    Dr. Rolph, I'd like to call your attention to 

15    PX 6.

16         A.    Okay.  I've got it in front of me now.

17         Q.    Is this the report which you've just been 

18    discussing?

19         A.    Yes, this is.

20         Q.    And what's the conclusion of this report about -- 

21    I'm sorry.  

22               What's the conclusion of the report?

23         A.    First of all, we reaffirmed our recommendation to 

24    the Bureau that they go ahead and plan for an adjustment in 

25    the 1990 Census, and, in particular, recommended that 
                                                              761

 1    adequate funds be provided for a large scale post- 

 2    enumeration survey and the other activities that would be 

 3    needed to carry out such an adjustment; and, secondly, we 

 4    recommended that an adjustment be made if there were no 

 5    obstacles to doing so.  

 6               MR. SHERMAN:  At this time,, your Honor, I move 

 7    PX 6 into evidence.  

 8               MR. BARON:  I didn't hear Dr. Rolph specifically 

 9    identify that he participated in the writing of this letter.

10               THE COURT:  Did you participate in the writing of 

11    this letter?

12               THE WITNESS:  Yes.  We -- if I could answer, the 

13    panel discussed what recommendations to make as part of our 

14    report to the director of the Census Bureau, and as best I 

15    can recall, drafted the language of this letter.  There may 

16    have been some comments changed before it finally went out, 

17    but the substance of the letter was approved as a whole by 

18    the panel.

19               MR. BARON:  Subject to that, we have no 

20    objection.

21               THE COURT:  Plaintiffs' Exhibit 6 is admitted.

22               (Exhibit PX 6 for identification was received in 

23    evidence) 

24    BY MR. SHERMAN:

25         Q.    Dr. Rolph, what happened to the National Academy 
                                                              762

 1    panel after that?

 2         A.    I guess the background is that this letter was 

 3    written in May of 1987.

 4               In October of 1987, then Under Secretary 

 5    Commerce, Robert Ortner, announced that there would not be a 

 6    correction, a statistical correction for the undercount in 

 7    1990.

 8               Subsequent to that decision, that is, in October 

 9    of '87, the panel's work and, indeed, the Bureau's work 

10    adjustment slowed down -- indeed, down to a halt.  And, in 

11    addition, the panel found that it was not able to get access 

12    to either Census Bureau documents or to Census Bureau's 

13    staff on the -- that related to adjustment or, more 

14    specifically, what was relevant then to coverage evaluation.

15               As a result, we completed our work, given 

16    recommendation on the experimental programs that I mentioned 

17    a moment ago; and, in the subsequent year, the panel decided 

18    that it could no longer carry out its mandate because of 

19    lack of access to the relevant information.

20               We wrote a letter, or had our chair, I should 

21    say, write a letter to the parent committee panel -- that is 

22    the Committee on National Statistics, that part of the 

23    academy, and indeed originally set up the panel -- indeed, 

24    Professor Fienberg, at the time the first panel started, was 

25    the TARO committee, and Dr. Bailar was the person in the 
                                                              763

 1    Census Bureau that sort of worked out the arrangement. 

 2               MR. BARON:  Your Honor, I move to strike that 

 3    portion of the answer where Dr. Rolph is speculating about 

 4    what the Census Bureau did, or why it acted in that 

 5    particular matter.  He doesn't know what the Census Bureau 

 6    did.

 7               THE COURT:  All right.  We'll strike that portion 

 8    of the answer.  

 9    BY MR. SHERMAN:

10         Q.    Well, Dr. Rolph, did the Census Bureau ever 

11    indicate to the panel that it no longer found it useful?

12         A.    Certainly not in any direct conversations that we 

13    had with any Census Bureau staff was that thought conveyed, 

14    no.

15         Q.    Do you know why the panel no longer was receiving 

16    cooperation from the Census Bureau?

17         A.    I have an understanding of that, of why it was, 

18    and my understanding is that -- 

19               MR. BARON:  Objection.  That's speculative.

20               THE COURT:  Sustained.  Hearsay.  

21    BY MR. SHERMAN:

22         Q.    Dr. Rolph, have you read Secretary Mosbacher's 

23    decision against adjustment?

24         A.    Yes, I have.

25         Q.    Is there anything in the Secretary's decision 
                                                              764

 1    which evidences consideration of the work of the National 

 2    Academy panel over its five-year existence?

 3         A.    I can recall only two minor footnotes that relate 

 4    to our panel's work.

 5         Q.    Is there anything in the Secretary's decision 

 6    which evidences a consideration of the recommendations made 

 7    by the two National Academy panels?

 8         A.    No, there isn't.  Indeed, I really found it quite 

 9    surprising that there wasn't because one of our explicit 

10    recommendations, as I mentioned a moment ago, regarding the 

11    question of loss functions and how to make an evaluation of 

12    whether or not to use -- to report corrected counts as 

13    opposed to uncorrected counts.  

14               And, as I said a few moments ago, we specifically 

15    recommended that counting up political jurisdictions not be 

16    the method of making such a judgment.  And, indeed, that's 

17    exactly what the Secretary did, which I would have thought 

18    he would have at least acknowledged that there was some 

19    difference of opinions as to whether that was the 

20    appropriate way to make such a decision.

21         Q.    Well -- 

22               MR. BARON:  Your Honor, the witness is 

23    speculating as to what the Secretary did.

24               THE COURT:  As to what?  

25               MR. BARON:  -- the Secretary did.  This witness 
                                                              765

 1    doesn't know what the Secretary did.  

 2               MR. SHERMAN:  Your Honor, I think what the 

 3    witness was speaking about is what's embodied in the 

 4    Secretary's decision document.

 5               THE COURT:  Yes.  The objection is overruled.  

 6    BY MR. SHERMAN:

 7         Q.    Dr. Rolph, just to clarify what you've just been 

 8    talking about, would you explain briefly what loss functions 

 9    are?

10         A.    Yes.  Yes, statisticians -- as, indeed, I 

11    suppose, anybody -- are interested in measuring the error 

12    one makes in a particular estimate.  And a particular error 

13    one is interested in measuring, how close the corrected 

14    counts are to the truth as compared to how close the 

15    uncorrected counts are.

16               The yardstick we use generally goes under the 

17    term of loss function.  And loss functions are a numerical 

18    measure of the accuracy of the estimation procedure.  And 

19    that is why their central focus is much in deliberation.

20         Q.    So in your view, are loss functions important to 

21    decisions on adjustment?

22         A.    Yes, I believe they are.

23         Q.    And, again, what type of loss function -- what 

24    were the panel's recommendations on loss functions?

25         A.    The panel recommended -- rather than counting 
                                                              766

 1    political jurisdictions, what the panel recommended was when 

 2    evaluating a geographical error, the weight given to that 

 3    area be comported to the population of the area, sort of, if 

 4    you will, a one-man one-vote principle.

 5         Q.    And did the panel make other recommendations 

 6    about loss functions?

 7         A.    Yes, we did.  We also recommended that the Bureau 

 8    use a variety of different loss functions in evaluating the 

 9    corrected versus the uncorrected counts.

10         Q.    And -- I'm sorry.  Is there more?

11         A.    And, finally, as I think I mentioned a few 

12    minutes ago, we also recommended that, in particular, that 

13    some of those loss functions be aimed specifically at 

14    measuring distributative accuracy.

15         Q.    Did the Bureau use loss functions to test the 

16    accuracy of adjustment against the enumeration in 1990?

17         A.    Yes, it did.

18         Q.    Were the loss functions produced by the Bureau 

19    consistent with the recommendations of the National Academy 

20    panel?

21         A.    Generally speaking, I would say yes.

22         Q.    And have you read the Secretary's interpretation 

23    of the loss function results?

24         A.    Yes, I have.

25         Q.    And how does that square with the recommendations 
                                                              767

 1    of the National Academy panel?

 2         A.    Well, as I indicated a moment ago, he, or the 

 3    Secretary's decision document, as I read it, looks at 

 4    political jurisdictions and counts.  That makes a numerical 

 5    count of which ones are better off versus worse off with 

 6    respect to their adjustment.

 7         Q.    So is that contrary to the recommendation of the 

 8    National Academy panel?

 9         A.    Yes, that's directly in conflict.

10         Q.    Was there any reference in the decision document 

11    to that recommendation of the National Academy panel?

12         A.    No, that was not the subject of one of the two 

13    footnotes I referred to earlier.

14         Q.    Dr. Rolph, would you describe the progress made 

15    by the Bureau on the issue of adjustment over the course of 

16    the National Academy panels' -- both panels' existence?

17         A.    Yes.  When we performed in early 1984, the Bureau 

18    -- I may have the timing off just slightly -- the Bureau was 

19    just in the process of forming a group to work on the 

20    undercount, a group of statisticians.  And it was Howard 

21    Hogan at the time who was the head of that group.  

22               Over the five-year existence of these two panels, 

23    the Bureau developed a series of research projects which 

24    they carried out, and a series of tests of carrying out an 

25    adjustment; and by the time the panel made its last 
                                                              768

 1    recommendation, as I indicated a few minutes ago, we had 

 2    come to the view that the Bureau had a process under way 

 3    that if all went as expected would lead to a successful 

 4    adjustment.  

 5               MR. SHERMAN:  Your Honor, I see that it's 5:00 

 6    and this would be a logical breaking point for me.

 7               THE COURT:  All right.  We will recess until 

 8    tomorrow morning at 9, I understand?  

 9               MR. BARON:  May I ask a question of counsel, to 

10    get a representation that this witness will be on the stand 

11    all day on Friday so that I'd like to know from the --

12               THE COURT:  We can't hear you.  

13               MR. BARON:  I'd like to know whether Professor 

14    Estrada would be called as a witness tomorrow, whether 

15    counsel intends that this witness will be examined in some 

16    length through the day.  

17               (Pause) 

18               MR. RIFKIND:  I think, as I'm bridging two chores 

19    here, it's a little hard for us to answer that question this 

20    evening.  We hope to put Professor Estrada on tomorrow.  

21    Part of that is in Mr. Baron's hands.  And, on the other 

22    hand, we don't want to start with him and not get finished 

23    with him since he comes from California.  We'll try to do a 

24    little recalibration this evening and report to you first 

25    thing in the morning?  
                                                              769

 1               Report to you this evening with the answer?  

 2               MR. BARON:  I think this evening will be better.  

 3               MR. RIFKIND:  We'll call you at your hotel this 

 4    evening.

 5               THE COURT:  Very good.  Okay.  See you tomorrow.              

 6               (Proceedings adjourned to 9 a.m., Friday, May 

 7    15th, 1992) 

 8    

 9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25
