Interface between Information Theory and Estimation Theory: THEORY AND APPLICATIONS

SPEAKER: Prof. Antonia Tulino, University of NaplesABSTRACT: For signals observed in Gaussian noise, there are several interesting intersections between information theory and linear and nonlinear minimum mean-square error (MMSE) estimation.A recently unveiled fundamental relationship between the input-output mutual information and the MMSE achievable by the optimal estimator of the input is now emerging as a powerful new tool that provides with explicit expressions for the sought derivative of the mutual information. This relationship holds for arbitrarily distributed scalar and vector signals, as well as for discrete-input discrete-output channels.Using this new tool, we have been able to analytically solve some of the long-standing constrained optimization problems in information theory. One example of such optimization problems is the problem often encountered in transmitter design of allocating a certain amount of power among a bank of parallel noninteracting channels. Examples abound, both in the wireline and the wireless domains are:* Multicarrier transmission. Signalling takes place over a number of distinct frequency bands, each of which constitutes a parallel channel. These bands may be nonoverlapping or, as in OFDM (orthogonal frequency division multiplexing), overlapping but with spectral shapes designed to ensure orthogonality. A prime application in the wireline world is the DMT (discrete multitone) technique employed in digital subscriber lines.* Multiantenna communication. If multiple transmit and receive antennas are employed and the transmitter knows the transfer coefficients between the antennas, the left singular vectors of the resulting matrix can be used for signaling and the right singular vectors for reception. The outcome is a set of parallel noninteracting channels.* Power control for fading channels. When the gain of an individual frequency-flat channel varies over time, it can be seen as a collection of parallel channels where each such channel encompasses a group of symbols over which the fading coefficients are identical.* Dispersive channels. For linear dispersive channels or parallel channels with correlated noises, a power-preserving orthonormal linear transformation at transmitter and receiver turns the channel into one with parallel branches with uncorrelated noises and possibly different signal-to-noise ratios.Another example of problems that use this fundamental link between information theory and estimation theory we have been able to analytically solve is the monotonic Decrease of the Non-Gaussianness of the Sum of independent Random Variables. Although long suspected that the non-Gaussianness decreases at each convolution, it was not shown until 2004 (in the equivalent version of increasing differential entropy) by Artstein, Ball, Barthe and Naor by means of a tour-de-force in functional analysis. Using the relationship between non-Gaussianness and MMSE in Gaussian channels, we will give a simplified proof of the above result dealing with the more general setting of non-identically distributed random variables.Bio:
Antonia Maria Tulino was born in Napoli, Italy, on September 12, 1971. She received the Dr. Engr. degree (summa cum laude) from the Universita degli Studi di Napoli Federico II, Napoli, Italy, in 1995 and the Ph.D. degree in electronic engineering from the Seconda Universita degli Studi di Napoli, Napoli, Italy, in 1999. In 1999, she was a Research Scientist at the Center for Wireless Communications (CWC), Oulu, Finland. From January 2000 to February 2001, she was a post-doctoral visitor with Princeton University, Princeton, NJ. From February 2001 to November 2002, she was an Assistant Professor with the Dipartmento di Ingegneria delle Telecomunicazioni, Universita degli Studi del Sannio, Benevento, Italy. From November 2002 she has been an Associate Professor with the Dipartmento di Ingegneria Elettronica e delle Telecomunicazioni, Universita degli Studi di Napoli "Federico II", Napoli, Italy. She is periodically appointed as a visiting research staff member in the Department of Electrical Engineering, Princeton University. Her current research interests are in the areas of statistical signal processing, information theory, and random matrix theory.Host: Prof. Giuseppe Caire, caire@usc.edu

Location: Hughes Aircraft Electrical Engineering Center (EEB) - 248

Audiences: Everyone Is Invited

Contact: Mayumi Thrasher