Figures for "An adaptive algorithm for unsupervised learning"



This supplementary information presents :

  • first, the code to generate the figures from the paper,
  • second, some control experiments that were mentionned in the paper,
  • finally, some perspectives for future work inspired by the algorithms presented in the paper.
In [1]:
%matplotlib inline
In [2]:
%load_ext autoreload
%autoreload 2

A convenience script model.py allows to run and cache most learning items in this notebooks:

In [3]:
%run model.py
tag = HULK
n_jobs = 0
In [4]:
from shl_scripts.shl_experiments import SHL
shl = SHL(**opts)
data = shl.get_data(matname=tag)
In [5]:
shl?
Type:        SHL
String form: <shl_scripts.shl_experiments.SHL object at 0x1134c3910>
File:        ~/science/HULK/SparseHebbianLearning/shl_scripts/shl_experiments.py
Docstring:  
Base class to define SHL experiments:
    - initialization
    - coding and learning
    - visualization
    - quantitative analysis
In [6]:
print('# of pixels per patch =', shl.patch_width**2)
# of pixels per patch = 441
In [7]:
print('number of patches, size of patches = ', data.shape)
print('average of patches = ', data.mean(), ' +/- ', data.mean(axis=1).std())
SE = np.sqrt(np.mean(data**2, axis=1))
print('average energy of data = ', SE.mean(), '+/-', SE.std())
number of patches, size of patches =  (65520, 441)
average of patches =  -4.1888600727021664e-05  +/-  0.006270387629074682
average energy of data =  0.26082782604823146 +/- 0.07415089441760706
In [8]:
#!ls -l {shl.cache_dir}/
#!ls -l {shl.cache_dir}/{tag}*
#!ls -ltr {shl.cache_dir}/{tag}*lock*
#!rm {shl.cache_dir}/{tag}*lock*
#!rm {shl.cache_dir}/{tag}*
#!rm {shl.cache_dir}/{tag}*HAP_seed*
#!ls -l {shl.cache_dir}/{tag}*
#!ls -ltr {shl.cache_dir}/{tag}*lock*

figure 1: Role of homeostasis in learning sparse representations

In [9]:
fname = 'figure_map'
# we cross-validate with 10 different learnings
one_cv = 8 # and pick one at random to display intermediate results

learning

The actual learning is done in a second object (here dico) from which we can access another set of properties and functions (see the shl_learn.py script):

In [10]:
homeo_methods = ['None', 'OLS', 'HEH']

list_figures = ['show_dico', 'time_plot_error', 'time_plot_logL', 'time_plot_MC', 'show_Pcum']
list_figures = []
dico = {}
for i_cv in range(N_cv):
    dico[i_cv] = {}
    for homeo_method in homeo_methods:
        shl = SHL(homeo_method=homeo_method, seed=seed+i_cv, **opts)
        dico[i_cv][homeo_method] = shl.learn_dico(data=data, list_figures=list_figures, matname=tag + '_' + homeo_method + '_seed=' + str(seed+i_cv))
In [11]:
list_figures = ['show_dico']
for i_cv in [one_cv]:
    for homeo_method in homeo_methods:
        print(hl + hs + homeo_method[:3] + hs + hl)
        shl = SHL(homeo_method=homeo_method, seed=seed+i_cv, **opts)
        shl.learn_dico(data=data, list_figures=list_figures, matname=tag + '_' + homeo_method + '_seed=' + str(seed+i_cv))

        print('size of dictionary = (number of filters, size of imagelets) = ', dico[i_cv][homeo_method].dictionary.shape)
        print('average of filters = ',  dico[i_cv][homeo_method].dictionary.mean(axis=1).mean(), 
              '+/-',  dico[i_cv][homeo_method].dictionary.mean(axis=1).std())
        SE = np.sqrt(np.sum(dico[i_cv][homeo_method].dictionary**2, axis=1))
        print('average energy of filters = ', SE.mean(), '+/-', SE.std())
        plt.show()
----------          Non          ----------
size of dictionary = (number of filters, size of imagelets) =  (676, 441)
average of filters =  -7.038410893224767e-06 +/- 0.0008419021277793694
average energy of filters =  1.0 +/- 3.866729645080236e-17
----------          OLS          ----------
size of dictionary = (number of filters, size of imagelets) =  (676, 441)
average of filters =  1.8871620517154972e-05 +/- 0.0007995474521857563
average energy of filters =  1.0 +/- 4.0734048673293375e-17