Dictionary Learning Algorithms

Algorithms

class dictlearn._dictionary_learning.DictionaryLearning(n_components=None, max_iter=1000, fit_algorithm='aksvd', transform_algorithm='omp', n_nonzero_coefs=None, code_init=None, dict_init=None, verbose=False, random_state=None, kernel_function=None, params=None, data_sklearn_compat=True)

Dictionary learning

Parameters
  • n_components (int, default=n_features) – Number of dictionary columns (atoms).

  • max_iter (int, default=100) – Maximum number of iterations.

  • fit_algorithm (string, default='aksvd') –

    Algorithm for dictionary optimization:

    • Standard DL: ‘ksvd’, ‘aksvd’, ‘uaksvd’, ‘sgk’, ‘nsgk’, ‘mod’

    • Regularized DL: ‘ksvd_reg’, ‘aksvd_reg’, ‘uaksvd_reg’, ‘sgk_reg’, ‘nsgk_reg’, ‘mod_reg’

    • Incoherent DL: ‘ksvd_coh’, ‘aksvd_coh’, ‘uaksvd_coh’, ‘sgk_coh’, ‘nsgk_coh’

    • Parallel Standard DL: ‘pksvd’, ‘paksvd’, ‘puaksvd’, ‘psgk’, ‘pnsgk’

    • Parallel Regularized DL: ‘pksvd_reg’, ‘paksvd_reg’, ‘puaksvd_reg’, ‘psgk_reg’, ‘pnsgk_reg’

    • Incoherent DL: ‘pksvd_coh’, ‘paksvd_coh’, ‘puaksvd_coh’, ‘psgk_coh’, ‘pnsgk_coh’

    • Kernel DL: ‘ker_ksvd’, ‘ker_aksvd’, ‘ker_uaksvd’, ‘ker_sgk’, ‘ker_nsgk’, ‘ker_mod’

    • Regularized Kernel DL: ‘ker_ksvd_reg’, ‘ker_aksvd_reg’, ‘ker_uaksvd_reg’, ‘ker_sgk_reg’, ‘ker_nsgk_reg’

    • Incoherent Kernel DL: ‘ker_ksvd_coh’, ‘ker_aksvd_coh’, ‘ker_uaksvd_coh’, ‘ker_sgk_coh’, ‘ker_nsgk_coh’

    • Parallel Kernel DL: ‘ker_pksvd’, ‘ker_paksvd’, ‘ker_puaksvd’, ‘ker_psgk’, ‘ker_pnsgk’

    • Parallel Regularized Kernel DL: ‘ker_pksvd_reg’, ‘ker_paksvd_reg’, ‘ker_puaksvd_reg’, ‘ker_psgk_reg’, ‘ker_pnsgk_reg’

    • Parallel Incoherent Kernel DL: ‘ker_pksvd_coh’, ‘ker_paksvd_coh’, ‘ker_puaksvd_coh’, ‘ker_psgk_coh’, ‘ker_pnsgk_coh’

    • Custom algorithm: If you want to propose a personal method of updating atoms you can use a handle function. For linear cases the handle function should respect the template custom_fit_algorithm(Y, D, X, params) and ker_custom_fit_algorithm(K, A, X, params) for the nonlinear cases.

  • transform_algorithm ({'omp'}, default='omp') –

    Algorithm used for computing the sparse representations:

    • ’omp’: Orthogonal Matching Pursuit.

    • custom algorithm: If you want to propose a personal method for computing the sparse representation you can use a handle function respecting the template custom_transform_algorithm(Y, D, n_nonzero_coefs). The handle function should return X, the sparse codes, and err, the approximation error.

  • n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm=’omp’. If None, then n_nonzero_coefs=int(n_features / 10).

  • code_init (ndarray of shape (n_samples, n_components), default=None) – Initial sparse representations matrix, for warm restart. Only used if dict_init is None.

  • dict_init (ndarray of shape (n_components, n_features), default=None) – Initial dictionary, for warm restart. Only used if code_init is None.

  • verbose (bool, default=False) – To control the verbosity of the procedure.

  • random_state (int, RandomState instance or None, default=None) – Used for initializing the dictionary when dict_init is not specified. Pass an int for reproducible results across multiple function calls.

  • kernel_function ({'rbf', 'poly'}, default='rbf') – Function used to compute the kernel matrix. If no kernel is provided then the standard dictionary learning algorithm will be used.

  • params (None, default=None) –

    Dictionary with provided parameters for the DL problem. If the parameters are not given, then they will be automatically initialized.

    • ’threshold_svd’ - threshold used to approximate singular values with zero, when solving a linear system (default = 1e-8)

    • ’atom_norm_tolerance’ - if an atom norm is less than this threshold, then it is considered zero (default = 1e-10)

    • ’reg_flag’ - flag used for regularization problems (default = False)

    • ’replatoms’ - unused dictionary atoms replacement strategy, (default = ReplAtoms.RANDOM)

      • ’ReplAtoms.ZERO’ : returns a zero column

      • ’ReplAtoms.RANDOM’ : returns a randomly generated atom

      • ’ReplAtoms.NO’ : perform no replacement

      • ’ReplAtoms.WORST’ : replace the worst represented signal

    • ’n_patoms’ - the number of atoms updated in parallel for parallel dictionary update strategies (default = None, meaning all atoms)

    • ’gamma’ - weight of coherence term in the Incoherent DL problem (default = 6)

    • ’mutual_coh_ipr’ - desired mutual coherence for iterative projections and rotations (default = 0.2)

    • ’n_it_ipr’ - number of iterations for iterative projections and rotations (default = 0)

    • ’sigma’ - value used for random Fourier features approximation (default = 1)

    • ’ff’ - forgetting factor value used for online strategies (default = 0.99)

    • ’regmu’ - weight of the regularization term in the Regularized DL problem (default = 0.1)

    • ’regvanish’ - value of vanishing factor in the Regularized DL problem (default = 0.95); after each iteration, regmu takes the value regmu*regvanish

    • ’regstop’ - maximum number of iterations in the Regularized DL problem (default = np.Inf); after regstop iterations, regmu becomes zero

  • data_sklearn_compat (bool, default=True) – Flag that establishes the compatibility of the data with the sklearn standard. When True, data should have dimensions (n_samples, n_features); otherwise, dimensions are (n_features, n_samples). Based on this value, the resulting dictionary will be a ndarray of shape (n_components, n_features) or (n_features, n_components).

D_

Trained dictionary

Type

ndarray of shape (n_components,n_features)

error_

Vector of RMSE values \(\|\mathbf{Y} - \mathbf{DX}\|_F/ \sqrt{mN}\) at each iteration

Type

array

error_extra_

Vector of errors at each iteration; the meaning depends on the DL objective function

Type

array

Example:

import matplotlib.pyplot as plt

from dictlearn import DictionaryLearning
from sklearn.datasets import make_sparse_coded_signal

n_components = 50      # number of atoms
n_features = 20        # signal dimension
n_nonzero_coefs = 3    # sparsity
n_samples = 100        # number of signals
n_iterations = 20      # number of dictionary learning iterations

max_iter = 10
fit_algorithm = "aksvd"
transform_algorithm = "omp"

Y, D_origin, X_origin = make_sparse_coded_signal(
    n_samples=n_samples,
    n_components=n_components,
    n_features=n_features,
    n_nonzero_coefs=n_nonzero_coefs,
    random_state=0
)

dl = DictionaryLearning(
    n_components=n_components,
    max_iter=max_iter,
    fit_algorithm=fit_algorithm,
    transform_algorithm=transform_algorithm,
    n_nonzero_coefs=n_nonzero_coefs,
    code_init=None,
    dict_init=None,
    verbose=False,
    random_state=None,
    kernel_function=None,
    params=None,
    data_sklearn_compat=False
)

dl.fit(Y)

plt.plot(range(max_iter), dl.error_, label=fit_algorithm)
plt.legend()
plt.show()

Notes

References: Bogdan Dumitrescu and Paul Irofti. Dictionary learning algorithms and applications. Springer, 2018.

fit(Y)

Fit the model from data in Y.

Parameters

Y (array-like of shape (n_samples, n_features)) – Training matrix, where n_samples in the number of samples and n_features is the number of features. Using the configuration variable data_sklearn_compat the matrix dimension will be updated.

Returns

self – Returns the object itself.

Return type

object

dictlearn._dictionary_learning.dictionary_learning(Y, D, X, max_iter, fit_algorithm, transform_algorithm, n_nonzero_coefs, params)

Standard Dictionary Learning

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • max_iter (int, default=100) – Maximum number of iterations to perform.

  • fit_algorithm (handle function) – Fit algorithm function.

  • transform_algorithm (handle function) – Transform algorithm function.

  • n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm=’omp’. If None, then n_nonzero_coefs=int(n_features / 10).

  • params (dict) – Dictionary with provided parameters for the DL problem.

Returns

  • D (ndarray of shape (n_features, n_components).) – The learned dictionary

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • rmse (float) – The approximation error for each iteration.

  • error_extra (float) – The extra error for each iteration.

dictlearn._dictionary_learning.kernel_dictionary_learning(Y, A, X, max_iter, fit_algorithm, transform_algorithm, n_nonzero_coefs, kernel_function, params)

Kernel Dictionary Learning

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • max_iter (int, default=100) – Maximum number of iterations to perform.

  • fit_algorithm (handle function) – Fit algorithm function.

  • transform_algorithm (handle function) – Transform algorithm function.

  • n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm=’omp’. If None, then n_nonzero_coefs=int(n_features / 10).

  • kernel_function (handle function) – Computes a kernel matrix.

  • params (dict) – Dictionary with provided parameters for the DL problem.

Returns

  • A (ndarray of shape (n_samples, n_components).) – The learned dictionary

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • rmse (float) – The approximation error for each iteration.

  • error_extra (float) – The extra error for each iteration.

dictlearn._dictionary_learning.online_dictionary_learning(Y, D, X, max_iter, fit_algorithm, transform_algorithm, n_nonzero_coefs, params)

Online Dictionary Learning

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • max_iter (int, default=100) – Maximum number of iterations to perform.

  • fit_algorithm (handle function) – Fit algorithm function.

  • transform_algorithm (handle function) – Transform algorithm function.

  • n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm=’omp’. If None, then n_nonzero_coefs=int(n_features / 10).

  • params (dict) – Dictionary with provided parameters for the DL problem.

Returns

  • D (ndarray of shape (n_features, n_components).) – The learned dictionary

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • rmse (float) – The approximation error for each iteration.

  • error_extra (float) – The extra error for each iteration.

dictlearn._dictionary_learning.sparse_encode(Y, D, algorithm='omp', n_nonzero_coefs=None, verbose=0)

Sparse coding

Each column of the result is the solution to a sparse coding problem. The goal is to find a sparse array code such that:

Y ~= D * X
Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • algorithm ({'omp'}, default='omp') –

    The algorithm for computing sparse representations:

    • ’omp’: Orthogonal Matching Pursuit;

  • n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. If None, then n_nonzero_coefs=int(n_features / 10).

  • verbose (int, default=0) – Controls the verbosity; the higher, the more messages.

Returns

  • X (ndarray of shape (n_components, n_samples)) – The sparse representations matrix

  • err (float) – The approximation error

Methods

In order to compare all the available methods we propose the following example

import copy
import numpy as np
import matplotlib.pyplot as plt

from dictlearn import DictionaryLearning
from sklearn.datasets import make_sparse_coded_signal

#############################################################################

n_components = 50      # number of atoms
n_features = 20        # signal dimension
n_nonzero_coefs = 3    # sparsity
n_samples = 100        # number of signals
n_iterations = 20      # number of dictionary learning iterations

#############################################################################

max_iter = 10
fit_algorithms = ["ksvd", "aksvd", "uaksvd", "sgk", "nsgk", "mod"]
transform_algorithm = "omp"

Y, D_origin, X_origin = make_sparse_coded_signal(
    n_samples=n_samples,
    n_components=n_components,
    n_features=n_features,
    n_nonzero_coefs=n_nonzero_coefs,
    random_state=0
)

D0 = np.random.randn(
    D_origin.shape[0],
    D_origin.shape[1]
)

for fit_algorithm in fit_algorithms:
    dl = DictionaryLearning(
        n_components=n_components,
        max_iter=max_iter,
        fit_algorithm=fit_algorithm,
        transform_algorithm=transform_algorithm,
        n_nonzero_coefs=n_nonzero_coefs,
        code_init=None,
        dict_init=copy.deepcopy(D0),
        verbose=False,
        random_state=None,
        kernel_function=None,
        params=None
    )

    dl.fit(Y)
    plt.plot(range(max_iter), dl.error_, label=fit_algorithm)

plt.legend()
plt.show()

The error evolution durring training stages is available as following

Standard Dictionary Learning

Standard Dictionary Learning

The Standard Dictionary Learning consists of optimizing

\[\begin{split}\begin{align} \min _{D, \mathbf{X}} & \|\mathbf{Y}-\mathbf{D} \mathbf{X}\|_{F}^{2} \\ \text { s.t. } & \left\|\mathbf{x}_{\ell}\right\|_{0} \leq s, \ell=1: N \\ & \left\|\mathbf{d}_{j}\right\|=1, j=1: n \label{eq-dl} \end{align}\end{split}\]

where \(\mathbf{Y} \in \mathbb{R}^{m \times N}\) is a given set of signals, represented by \(N\) signals of size \(m\), \(\mathbf{D} \in \mathbb{R}^{m \times n}\) is the trained dictionary, \(\mathbf{X} \in \mathbb{R}^{n \times N}\) is the sparse representation and \(s\) represents the sparsity constraint.

dictlearn.methods._ksvd.ksvd(Y, D, X, params)

K-SVD

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary with provided parameters for the DL problem.

Returns

  • D (ndarray of shape (n_features, n_components)) – The updated dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.aksvd(Y, D, X, params)

Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.uaksvd(Y, D, X, params)

Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.sgk(Y, D, X, params)

Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.nsgk(Y, D, X, params)

New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._mod.mod(Y, D, X, params)

Method of Optimal Directions. The parameters are the same as for K-SVD.

Regularized Dictionary Learning

Least squares regularization consists of optimizing

\[\min _{D, \mathbf{X}} \|\mathbf{Y}-\mathbf{D} \mathbf{X}\|_{F}^{2}+\mu\|\mathbf{X}\|_{F}^{2}\]

where \(\mu > 0\) is the regularization parameter.

dictlearn.methods._ksvd.ksvd_reg(Y, D, X, params)

Regularized K-SVD

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • D (ndarray of shape (n_features, n_components)) – The updated dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.aksvd_reg(Y, D, X, params)

Regularized Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.uaksvd_reg(Y, D, X, params)

Updated-error Regularized Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.sgk_reg(Y, D, X, params)

Regularized Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.nsgk_reg(Y, D, X, params)

Regularized New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._mod.mod_reg(Y, D, X, params)

Regularized Method of Optimal Directions. The parameters are the same as for K-SVD.

Incoherent Dictionary Learning

Dictionary Learning algorithms that jointly optimize the representation error and the coherence of the dictionary. The objective function consists in

\[\min _{D, X}\|\mathbf{Y}-\mathbf{D} \mathbf{X}\|_{F}^{2}+\gamma\left\|\mathbf{D}^{T} \mathbf{D}-\mathbf{I}\right\|_{F}^{2}\]

where \(\gamma > 0\) is a trade-off parameter.

dictlearn.methods._ksvd.ksvd_coh(Y, D, X, params)

K-SVD algorithm with coherence reduction

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • D (ndarray of shape (n_features, n_components)) – The updated dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.aksvd_coh(Y, D, X, params)

Approximate K-SVD algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.uaksvd_coh(Y, D, X, params)

Updated-error Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.sgk_coh(Y, D, X, params)

Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.nsgk_coh(Y, D, X, params)

New Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

Parallel Standard Dictionary Learning

Solves the Standard Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.pksvd(Y, D, X, params)

Parallel K-SVD algorithm

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • D (ndarray of shape (n_features, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.paksvd(Y, D, X, params)

Parallel Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.puaksvd(Y, D, X, params)

Parallel Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.psgk(Y, D, X, params)

Parallel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.pnsgk(Y, D, X, params)

Parallel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

Parallel Regularized Dictionary Learning

Solves the Regularized Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.pksvd_reg(Y, D, X, params)

Regularized Parallel K-SVD

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • D (ndarray of shape (n_features, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.paksvd_reg(Y, D, X, params)

Parallel Regularized Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.puaksvd_reg(Y, D, X, params)

Parallel Regularized Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.psgk_reg(Y, D, X, params)

Parallel Regularized Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.pnsgk_reg(Y, D, X, params)

Parallel Regularized New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

Parallel Incoherent Dictionary Learning

Solves the Incoherent Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.pksvd_coh(Y, D, X, params)

Parallel K-SVD with coherence reduction

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • D (ndarray of shape (n_features, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.paksvd_coh(Y, D, X, params)

Parallel Approximate K-SVD algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.puaksvd_coh(Y, D, X, params)

Parallel Updated-error Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.psgk_coh(Y, D, X, params)

Parallel Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.pnsgk_coh(Y, D, X, params)

Parallel New Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

Kernel Dictionary Learning

In order to evade the linear character of the representation, a Kernel version of the Dictionary Learning problem is introduced. Through this method, the space of signals is extended to a nonlinear feature vector space. We associate with each signal \(\mathbf{y}\) the feature vector \(\varphi(\mathbf{y})\), where \(\varphi(\mathbf{y})\) is a nonlinear function. The dictionary \(\mathbf{D}\) is also extended to a nonlinear space by \(\varphi(\mathbf{y})\mathbf{A}\), where \(\mathbf{A}\) contains the coefficients of the dictionary. So, the DL problem is transformed into

\[\begin{split}\begin{align} \min _{D, \mathbf{X}} & \|\varphi(\mathbf{Y})-\varphi(\mathbf{Y})\mathbf{A} \mathbf{X}\|_{F}^{2} \\ \text { s.t. } & \left\|\mathbf{x}_{\ell}\right\|_{0} \leq s, \ell=1: N \\ & \left\|\mathbf{a}_{j}\right\|=1, j=1: n \end{align}\end{split}\]
dictlearn.methods._ksvd.ker_ksvd(K, A, X, params)

Kernel K-SVD

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_aksvd(K, A, X, params)

Kernel Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_uaksvd(K, A, X, params)

Kernel Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_sgk(K, A, X, params)

Kernel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_nsgk(K, A, X, params)

Kernel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._mod.ker_mod(K, A, X, params)

Kernel Method of Optimal Directions. The parameters are the same as for K-SVD.

Regularized Kernel Dictionary Learning

A regularization version for the Kernel Dictionary Learning. The optimization problem consists in

\[\min _{A, \mathbf{X}} \|\varphi(\mathbf{Y})-\varphi(\mathbf{Y}) \mathbf{A} \mathbf{X}\|_{F}^{2}+\mu\|\mathbf{X}\|_{F}^{2}\]

where \(\mu > 0\) is the regularization parameter.

dictlearn.methods._ksvd.ker_ksvd_reg(K, A, X, params)

Regularized Kernel K-SVD

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_aksvd_reg(K, A, X, params)

Regularized Kernel Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_uaksvd_reg(K, A, X, params)

Regularized Kernel Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_sgk_reg(K, A, X, params)

Regularized Kernel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_nsgk_reg(K, A, X, params)

Regularized New Kernel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

Incoherent Kernel Dictionary Learning

A incoherent version for the Kernel Dictionary Learning. The optimization problem consists

\[\min _{A, X}\|\varphi(\mathbf{Y})-\varphi(\mathbf{Y}) \mathbf{A} \mathbf{X}\|_{F}^{2}+\gamma\left\|\mathbf{A}^{T} \varphi(\mathbf{Y})^{T} \varphi(\mathbf{Y}) \mathbf{A}-\mathbf{I}\right\|_{F}^{2}\]

where \(\gamma > 0\) is a trade-off parameter.

dictlearn.methods._ksvd.ker_ksvd_coh(K, A, X, params)

Kernel K-SVD with coherence reduction

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_aksvd_coh(K, A, X, params)

Regularized Kernel Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_uaksvd_coh(K, A, X, params)

Kernel Updated-error Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_sgk_coh(K, A, X, params)

Kernel Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_nsgk_coh(K, A, X, params)

Kernel New Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

Parallel Kernel Dictionary Learning

Solves the Kernel Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.ker_pksvd(K, A, X, params)

Parallel Kernel K-SVD

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_paksvd(K, A, X, params)

Parallel Kernel Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_puaksvd(K, A, X, params)

Parallel Kernel Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_psgk(K, A, X, params)

Parallel Kernel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_pnsgk(K, A, X, params)

Parallel Kernel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

Parallel Regularized Kernel Dictionary Learning

Solves the Regularized Kernel Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.ker_pksvd_reg(K, A, X, params)

Parallel Regularized Kernel K-SVD

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_paksvd_reg(K, A, X, params)

Parallel Regularized Kernel approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_puaksvd_reg(K, A, X, params)

Parallel Kernel Regularized Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_psgk_reg(K, A, X, params)

Parallel Regularized Kernel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_pnsgk_reg(K, A, X, params)

Parallel Regularized Kernel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

Parallel Incoherent Kernel Dictionary Learning

Solves the Incoherent Kernel Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.ker_pksvd_coh(K, A, X, params)

Parallel Kernel K-SVD with coherence reduction

Parameters
  • Y (ndarray of shape (n_features, n_samples)) – Data matrix.

  • A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

  • X (ndarray of shape (n_components, n_samples)) – The sparse codes.

  • params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

  • A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

  • X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_paksvd_coh(K, A, X, params)

Parallel Kernel Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_puaksvd_coh(K, A, X, params)

Parallel Kernel Updated-error Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_psgk_coh(K, A, X, params)

Parallel Kernel Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_pnsgk_coh(K, A, X, params)

Parallel Kernel New Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.