Welcome!

This is Lizhong Zheng. I am a professor in EECS at MIT. I work in the area of statistical data analysis. I make this page to keep some of the experiments and demos related to some of our recent research works. The goal is not only to show you the codes, but also some explanations on why we did it in the first place, together with a little math, as well as some pointers to help you to run and make it your own experiments. That got to be fun, ain't it! (This actually helps me to take a break from writing formal research papers, to write something casual instead, which is fun too!)

The main question we try to answer on this page is: 

Why are neural networks so powerful?

Well, that is a difficult question that has haunted us for about 30 years. The more successes we see from the applications of deep-learning, the more puzzled we are. Why is it that we can plug in this “thing” to pretty much any problem, classificationor prediction, and with just some limited amount of tuning, we almost always can get good results. While some of us are amazed by this power, particularly the universal applicability; some found it hard to be convinced without a deeper understanding.

Now here is an answer! Ready?

In short, Neural Networks extract from the data the most relevant part of the information that describes the statistical dependence between the features and the labels. In other words, the size of a Neural Networks specifies a data structurethat we can compute and store, and the result of training the network is the best approximation of the statistical relationship between the features and the labels that can be represented by this data structure.

aaa

import numpy as np 
import matplotlib.pyplot as plt 

from keras.models import Sequential 
from keras.layers import Dense, Activation 
from keras.optimizers import SGD
 
 X= np.zeros([N, Dx])  
    Labels=np.zeros([N, Cy]) # neural network takes the indicators instead of Y  
      
    for i in range(N):  
        X[i, :]=M[:, Y[i]]  
        Labels[i, Y[i]]=1  
      
    X=X+ np.random.normal(0, .5, X.shape)