\documentclass{article}
\usepackage[margin=1.2in]{geometry}
\usepackage{graphicx}
\usepackage{amsmath,amssymb,amsthm,bm}
\usepackage{latexsym,color,minipage-marginpar,caption,multirow,verbatim}
\usepackage{enumerate}
\usepackage{times}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\cP}{\mathcal{P}}
\newcommand{\cC}{\mathcal{C}}
\newcommand{\cX}{\mathcal{X}}
\newcommand{\ep}{\varepsilon}
\newcommand{\td}{\,\textrm{d}}
\newcommand{\simiid}{\overset{\textrm{i.i.d.}}{\sim}}
\newcommand{\simind}{\overset{\textrm{ind.}}{\sim}}
\newcommand{\toProb}{\overset{p}{\to}}
\newcommand{\toPtheta}{\overset{{P_\theta}}{\to}}
\newcommand{\Var}{\text{Var}}
\newcommand{\red}{\color{red}}
\definecolor{darkblue}{rgb}{0.2, 0.2, 0.5}
\newcommand{\sol}{~\\\color{darkblue}{\bf Solution:~\\}}
\begin{document}
\title{Stats 210A, Fall 2023\\
Homework 8\\
{\large {\bf Due date}: Wednesday, Oct. 25}}
\date{}
\maketitle
\vspace{-5em}
\begin{description}
\item[1. Directional error claims]\hfill\\
Suppose $\cP = \{P_{\theta}:\; \theta \in \RR\}$, and $T(X)$ is a continuous test statistic that is stochastically increasing in $\theta$, meaning
\[
\PP_{\theta_1}(T(X) \leq t) \leq \PP_{\theta_0}(T(X) \leq t), \quad \text{ for all } t\in\RR \text{ and } \theta_1>\theta_0.
\]
As we have discussed in class, this property guarantees that a one-tailed test of $H_0:\; \theta \leq 0$ vs. $H_1:\; \theta > 0$ that rejects for large values of $T(X)$, with cutoff $c$ chosen to solve $\PP_{0}(T(X) > c) = \alpha$, will give a valid level-$\alpha$ test over the whole null distribution.
Now let $\phi(X)$ represent any level-$\alpha$ {\em two-tailed} test of $H_0:\; \theta = 0$ vs. $H_1:\; \theta \neq 0$ that rejects for extreme values of $T(X)$.
Assume that we also always make a {\em directional claim} about $\text{sign}(\theta)$ whenever we reject $H_0$. That is, we make one of {\em three} decisions: if $T(X) \in [c_1,c_2]$ then we do not reject $H_0$ (and we make no claim about the sign of $\theta$); if $T(X) > c_2$, then we reject $H_0$ and claim further that $\theta > 0$; and if $T(X) < c_1$, then we reject $H_0$ and claim further that $\theta < 0$. So now there are two kinds of Type I errors we could make: we could reject $H_0$ when it is really true, {\em or} we could reject $H_0$ when it is really false but call the sign wrong. Show that we have control of the {\em directional error rate}:
\[
\sup_{\theta \in \RR} \PP_\theta(\text{False rejection or wrong sign call}) \leq \alpha
\]
{\bf Moral:} Having MLR or exponential families are nice to be able to talk about optimal tests, but stochastically increasing is a useful condition for getting valid tests in various contexts.
In particular, people often complain that we do not learn anything about $\theta$ by rejecting $H_0:\; \theta=0$, because we should have already known $\theta$ was not exactly zero. This line of argument ignores the fact that (in most testing settings) we can also draw a definite conclusion about the sign of $\theta$ whenever we reject $H_0:\;\theta = 0$, without inflating the error rate.
\item[2. Some two-tailed tests]\hfill\\
Consider testing $H_0:\,\theta = \theta_0$ vs $H_1:\,\theta \neq \theta_0$ in a one-parameter exponential family of the form $p_\theta(x) = e^{\theta T(x) - A(\theta)}h(x)$. We stated in class that among all {\em unbiased, level-$\alpha$} tests, the one that rejects for extreme (i.e., large or small) values of $T(X)$ is uniformly most powerful (simultaneously maximizes power for all alternatives).
The equal-tailed level-$\alpha$ test that rejects for extreme values of $T(X)$ does not satisfy as interesting an optimality property but it is also a competitive test. Depending on the distribution, the equal-tailed test and the UMPU test may or may not coincide.
Numerically find the equal-tailed and UMPU test for the following hypothesis testing problems at level $\alpha=0.05$. For each problem,
\begin{enumerate}[(i)]
\item derive the appropriate tests (leaving the cutoff values abstract),
\item numerically compute the cutoff values $c$ (no $\gamma$ necessary since these are continuous problems), and
\item invert the equal-tailed test to give an interval for the data value specified (no need to invert the unbiased test).
\end{enumerate}
\begin{enumerate}[(a)]
\item $X_i \simind N(\theta, \sigma_i^2)$ for $i=1,\ldots,n$, where $\sigma_i^2$ are known positive constants and $\theta \in \RR$ is unknown. Test $H_0:\; \theta = 0$ vs. $H_1:\; \theta \neq 0$, with $n = 20$ and $\sigma_i^2 = i$. On your power plot, also plot the power function of the (sub-optimal) test that rejects for extreme values of $\sum_i X_i$.
\item $X_1,\ldots, X_n \simiid \text{Pareto}(\theta) = \theta x^{-(1+\theta)}$, for $\theta > 0$ and $x > 1$ (also called a power law distribution). Test $H_0:\; \theta = 1$ vs. $H_1:\; \theta \neq 1$, for $n = 100$. On your power plot, also plot the power function of the (sub-optimal) test that rejects for large $\sum_i X_i$.
\end{enumerate}
\item[3. Maximizing average power]\hfill\\
In situations where there is not a UMP test, we cannot simultaneously maximize power for all alternatives. However, if the null is simple ($\Theta_0 = \{\theta_0\}$) and we have a prior $\Lambda_1$ over the alternative parameter space $\Theta_1$, we can maximize average power by rejecting for large values of:
\[
T(x) = \frac{\int_{\Theta_1}p_{\theta}(x)\td \Lambda_1(\theta)}{p_{\theta_0}(x)}
\]
Show that this test maximizes the average-case power $\int_{\Theta_1}\EE_\theta\phi(X)\td \Lambda_1(\theta)$ among all tests with level $\alpha$.
{\bf Hint:} Show that it can be viewed as a Neyman-Pearson test for a particular simple alternative.
\item[4. $p$-value densities]\hfill\\
Suppose $\cP$ is a family with monotone likelihood ratio in $T(X)$, and the distribution of $T(X)$ is continuous with common support for all $\theta$. Let $\phi_{\alpha}$ denote the UMP level-$\alpha$ test of $H_0:\theta\leq \theta_0$ vs. $H_0:\theta > \theta_0$ that rejects when $T(X)$ is large. Let $p(X)$ denote the resulting $p$-value. Show that $p(X)\sim \text{Unif}[0,1]$ if $\theta=\theta_0$, has non-increasing density on $[0,1]$ if $\theta>\theta_0$, and has non-decreasing density on $[0,1]$ if $\theta<\theta_0$.
{\bf Note:} As always there is some ambiguity in how we could define the density; to resolve this ambiguity note it is equivalent to show that the CDF is linear, concave, or convex, or you can define the density unambiguously as the derivative of the CDF.
\end{description}
\end{document}