# Dictionary Learning Algorithms

## Algorithms

class dictlearn._dictionary_learning.DictionaryLearning(n_components=None, max_iter=1000, fit_algorithm='aksvd', transform_algorithm='omp', n_nonzero_coefs=None, code_init=None, dict_init=None, verbose=False, random_state=None, kernel_function=None, params=None, data_sklearn_compat=True)

Dictionary learning

Parameters
• n_components (int, default=n_features) – Number of dictionary columns (atoms).

• max_iter (int, default=100) – Maximum number of iterations.

• fit_algorithm (string, default='aksvd') –

Algorithm for dictionary optimization:

• Standard DL: ‘ksvd’, ‘aksvd’, ‘uaksvd’, ‘sgk’, ‘nsgk’, ‘mod’

• Regularized DL: ‘ksvd_reg’, ‘aksvd_reg’, ‘uaksvd_reg’, ‘sgk_reg’, ‘nsgk_reg’, ‘mod_reg’

• Incoherent DL: ‘ksvd_coh’, ‘aksvd_coh’, ‘uaksvd_coh’, ‘sgk_coh’, ‘nsgk_coh’

• Parallel Standard DL: ‘pksvd’, ‘paksvd’, ‘puaksvd’, ‘psgk’, ‘pnsgk’

• Parallel Regularized DL: ‘pksvd_reg’, ‘paksvd_reg’, ‘puaksvd_reg’, ‘psgk_reg’, ‘pnsgk_reg’

• Incoherent DL: ‘pksvd_coh’, ‘paksvd_coh’, ‘puaksvd_coh’, ‘psgk_coh’, ‘pnsgk_coh’

• Kernel DL: ‘ker_ksvd’, ‘ker_aksvd’, ‘ker_uaksvd’, ‘ker_sgk’, ‘ker_nsgk’, ‘ker_mod’

• Regularized Kernel DL: ‘ker_ksvd_reg’, ‘ker_aksvd_reg’, ‘ker_uaksvd_reg’, ‘ker_sgk_reg’, ‘ker_nsgk_reg’

• Incoherent Kernel DL: ‘ker_ksvd_coh’, ‘ker_aksvd_coh’, ‘ker_uaksvd_coh’, ‘ker_sgk_coh’, ‘ker_nsgk_coh’

• Parallel Kernel DL: ‘ker_pksvd’, ‘ker_paksvd’, ‘ker_puaksvd’, ‘ker_psgk’, ‘ker_pnsgk’

• Parallel Regularized Kernel DL: ‘ker_pksvd_reg’, ‘ker_paksvd_reg’, ‘ker_puaksvd_reg’, ‘ker_psgk_reg’, ‘ker_pnsgk_reg’

• Parallel Incoherent Kernel DL: ‘ker_pksvd_coh’, ‘ker_paksvd_coh’, ‘ker_puaksvd_coh’, ‘ker_psgk_coh’, ‘ker_pnsgk_coh’

• Custom algorithm: If you want to propose a personal method of updating atoms you can use a handle function. For linear cases the handle function should respect the template custom_fit_algorithm(Y, D, X, params) and ker_custom_fit_algorithm(K, A, X, params) for the nonlinear cases.

• transform_algorithm ({'omp'}, default='omp') –

Algorithm used for computing the sparse representations:

• ’omp’: Orthogonal Matching Pursuit.

• custom algorithm: If you want to propose a personal method for computing the sparse representation you can use a handle function respecting the template custom_transform_algorithm(Y, D, n_nonzero_coefs). The handle function should return X, the sparse codes, and err, the approximation error.

• n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm=’omp’. If None, then n_nonzero_coefs=int(n_features / 10).

• code_init (ndarray of shape (n_samples, n_components), default=None) – Initial sparse representations matrix, for warm restart. Only used if dict_init is None.

• dict_init (ndarray of shape (n_components, n_features), default=None) – Initial dictionary, for warm restart. Only used if code_init is None.

• verbose (bool, default=False) – To control the verbosity of the procedure.

• random_state (int, RandomState instance or None, default=None) – Used for initializing the dictionary when dict_init is not specified. Pass an int for reproducible results across multiple function calls.

• kernel_function ({'rbf', 'poly'}, default='rbf') – Function used to compute the kernel matrix. If no kernel is provided then the standard dictionary learning algorithm will be used.

• params (None, default=None) –

Dictionary with provided parameters for the DL problem. If the parameters are not given, then they will be automatically initialized.

• ’threshold_svd’ - threshold used to approximate singular values with zero, when solving a linear system (default = 1e-8)

• ’atom_norm_tolerance’ - if an atom norm is less than this threshold, then it is considered zero (default = 1e-10)

• ’reg_flag’ - flag used for regularization problems (default = False)

• ’replatoms’ - unused dictionary atoms replacement strategy, (default = ReplAtoms.RANDOM)

• ’ReplAtoms.ZERO’ : returns a zero column

• ’ReplAtoms.RANDOM’ : returns a randomly generated atom

• ’ReplAtoms.NO’ : perform no replacement

• ’ReplAtoms.WORST’ : replace the worst represented signal

• ’n_patoms’ - the number of atoms updated in parallel for parallel dictionary update strategies (default = None, meaning all atoms)

• ’gamma’ - weight of coherence term in the Incoherent DL problem (default = 6)

• ’mutual_coh_ipr’ - desired mutual coherence for iterative projections and rotations (default = 0.2)

• ’n_it_ipr’ - number of iterations for iterative projections and rotations (default = 0)

• ’sigma’ - value used for random Fourier features approximation (default = 1)

• ’ff’ - forgetting factor value used for online strategies (default = 0.99)

• ’regmu’ - weight of the regularization term in the Regularized DL problem (default = 0.1)

• ’regvanish’ - value of vanishing factor in the Regularized DL problem (default = 0.95); after each iteration, regmu takes the value regmu*regvanish

• ’regstop’ - maximum number of iterations in the Regularized DL problem (default = np.Inf); after regstop iterations, regmu becomes zero

• data_sklearn_compat (bool, default=True) – Flag that establishes the compatibility of the data with the sklearn standard. When True, data should have dimensions (n_samples, n_features); otherwise, dimensions are (n_features, n_samples). Based on this value, the resulting dictionary will be a ndarray of shape (n_components, n_features) or (n_features, n_components).

D_

Trained dictionary

Type

ndarray of shape (n_components,n_features)

error_

Vector of RMSE values $$\|\mathbf{Y} - \mathbf{DX}\|_F/ \sqrt{mN}$$ at each iteration

Type

array

error_extra_

Vector of errors at each iteration; the meaning depends on the DL objective function

Type

array

Example:

import matplotlib.pyplot as plt

from dictlearn import DictionaryLearning
from sklearn.datasets import make_sparse_coded_signal

n_components = 50      # number of atoms
n_features = 20        # signal dimension
n_nonzero_coefs = 3    # sparsity
n_samples = 100        # number of signals
n_iterations = 20      # number of dictionary learning iterations

max_iter = 10
fit_algorithm = "aksvd"
transform_algorithm = "omp"

Y, D_origin, X_origin = make_sparse_coded_signal(
n_samples=n_samples,
n_components=n_components,
n_features=n_features,
n_nonzero_coefs=n_nonzero_coefs,
random_state=0
)

dl = DictionaryLearning(
n_components=n_components,
max_iter=max_iter,
fit_algorithm=fit_algorithm,
transform_algorithm=transform_algorithm,
n_nonzero_coefs=n_nonzero_coefs,
code_init=None,
dict_init=None,
verbose=False,
random_state=None,
kernel_function=None,
params=None,
data_sklearn_compat=False
)

dl.fit(Y)

plt.plot(range(max_iter), dl.error_, label=fit_algorithm)
plt.legend()
plt.show()


Notes

References: Bogdan Dumitrescu and Paul Irofti. Dictionary learning algorithms and applications. Springer, 2018.

fit(Y)

Fit the model from data in Y.

Parameters

Y (array-like of shape (n_samples, n_features)) – Training matrix, where n_samples in the number of samples and n_features is the number of features. Using the configuration variable data_sklearn_compat the matrix dimension will be updated.

Returns

self – Returns the object itself.

Return type

object

dictlearn._dictionary_learning.dictionary_learning(Y, D, X, max_iter, fit_algorithm, transform_algorithm, n_nonzero_coefs, params)

Standard Dictionary Learning

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• max_iter (int, default=100) – Maximum number of iterations to perform.

• fit_algorithm (handle function) – Fit algorithm function.

• transform_algorithm (handle function) – Transform algorithm function.

• n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm=’omp’. If None, then n_nonzero_coefs=int(n_features / 10).

• params (dict) – Dictionary with provided parameters for the DL problem.

Returns

• D (ndarray of shape (n_features, n_components).) – The learned dictionary

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• rmse (float) – The approximation error for each iteration.

• error_extra (float) – The extra error for each iteration.

dictlearn._dictionary_learning.kernel_dictionary_learning(Y, A, X, max_iter, fit_algorithm, transform_algorithm, n_nonzero_coefs, kernel_function, params)

Kernel Dictionary Learning

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• max_iter (int, default=100) – Maximum number of iterations to perform.

• fit_algorithm (handle function) – Fit algorithm function.

• transform_algorithm (handle function) – Transform algorithm function.

• n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm=’omp’. If None, then n_nonzero_coefs=int(n_features / 10).

• kernel_function (handle function) – Computes a kernel matrix.

• params (dict) – Dictionary with provided parameters for the DL problem.

Returns

• A (ndarray of shape (n_samples, n_components).) – The learned dictionary

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• rmse (float) – The approximation error for each iteration.

• error_extra (float) – The extra error for each iteration.

dictlearn._dictionary_learning.online_dictionary_learning(Y, D, X, max_iter, fit_algorithm, transform_algorithm, n_nonzero_coefs, params)

Online Dictionary Learning

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• max_iter (int, default=100) – Maximum number of iterations to perform.

• fit_algorithm (handle function) – Fit algorithm function.

• transform_algorithm (handle function) – Transform algorithm function.

• n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm=’omp’. If None, then n_nonzero_coefs=int(n_features / 10).

• params (dict) – Dictionary with provided parameters for the DL problem.

Returns

• D (ndarray of shape (n_features, n_components).) – The learned dictionary

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• rmse (float) – The approximation error for each iteration.

• error_extra (float) – The extra error for each iteration.

dictlearn._dictionary_learning.sparse_encode(Y, D, algorithm='omp', n_nonzero_coefs=None, verbose=0)

Sparse coding

Each column of the result is the solution to a sparse coding problem. The goal is to find a sparse array code such that:

Y ~= D * X

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• algorithm ({'omp'}, default='omp') –

The algorithm for computing sparse representations:

• ’omp’: Orthogonal Matching Pursuit;

• n_nonzero_coefs (int, default=None) – Number of nonzero coefficients to target in each column of the solution. If None, then n_nonzero_coefs=int(n_features / 10).

• verbose (int, default=0) – Controls the verbosity; the higher, the more messages.

Returns

• X (ndarray of shape (n_components, n_samples)) – The sparse representations matrix

• err (float) – The approximation error

## Methods

In order to compare all the available methods we propose the following example

import copy
import numpy as np
import matplotlib.pyplot as plt

from dictlearn import DictionaryLearning
from sklearn.datasets import make_sparse_coded_signal

#############################################################################

n_components = 50      # number of atoms
n_features = 20        # signal dimension
n_nonzero_coefs = 3    # sparsity
n_samples = 100        # number of signals
n_iterations = 20      # number of dictionary learning iterations

#############################################################################

max_iter = 10
fit_algorithms = ["ksvd", "aksvd", "uaksvd", "sgk", "nsgk", "mod"]
transform_algorithm = "omp"

Y, D_origin, X_origin = make_sparse_coded_signal(
n_samples=n_samples,
n_components=n_components,
n_features=n_features,
n_nonzero_coefs=n_nonzero_coefs,
random_state=0
)

D0 = np.random.randn(
D_origin.shape[0],
D_origin.shape[1]
)

for fit_algorithm in fit_algorithms:
dl = DictionaryLearning(
n_components=n_components,
max_iter=max_iter,
fit_algorithm=fit_algorithm,
transform_algorithm=transform_algorithm,
n_nonzero_coefs=n_nonzero_coefs,
code_init=None,
dict_init=copy.deepcopy(D0),
verbose=False,
random_state=None,
kernel_function=None,
params=None
)

dl.fit(Y)
plt.plot(range(max_iter), dl.error_, label=fit_algorithm)

plt.legend()
plt.show()


The error evolution durring training stages is available as following

### Standard Dictionary Learning

The Standard Dictionary Learning consists of optimizing

\begin{split}\begin{align} \min _{D, \mathbf{X}} & \|\mathbf{Y}-\mathbf{D} \mathbf{X}\|_{F}^{2} \\ \text { s.t. } & \left\|\mathbf{x}_{\ell}\right\|_{0} \leq s, \ell=1: N \\ & \left\|\mathbf{d}_{j}\right\|=1, j=1: n \label{eq-dl} \end{align}\end{split}

where $$\mathbf{Y} \in \mathbb{R}^{m \times N}$$ is a given set of signals, represented by $$N$$ signals of size $$m$$, $$\mathbf{D} \in \mathbb{R}^{m \times n}$$ is the trained dictionary, $$\mathbf{X} \in \mathbb{R}^{n \times N}$$ is the sparse representation and $$s$$ represents the sparsity constraint.

dictlearn.methods._ksvd.ksvd(Y, D, X, params)

K-SVD

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary with provided parameters for the DL problem.

Returns

• D (ndarray of shape (n_features, n_components)) – The updated dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.aksvd(Y, D, X, params)

Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.uaksvd(Y, D, X, params)

Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.sgk(Y, D, X, params)

Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.nsgk(Y, D, X, params)

New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._mod.mod(Y, D, X, params)

Method of Optimal Directions. The parameters are the same as for K-SVD.

### Regularized Dictionary Learning

Least squares regularization consists of optimizing

$\min _{D, \mathbf{X}} \|\mathbf{Y}-\mathbf{D} \mathbf{X}\|_{F}^{2}+\mu\|\mathbf{X}\|_{F}^{2}$

where $$\mu > 0$$ is the regularization parameter.

dictlearn.methods._ksvd.ksvd_reg(Y, D, X, params)

Regularized K-SVD

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• D (ndarray of shape (n_features, n_components)) – The updated dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.aksvd_reg(Y, D, X, params)

Regularized Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.uaksvd_reg(Y, D, X, params)

Updated-error Regularized Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.sgk_reg(Y, D, X, params)

Regularized Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.nsgk_reg(Y, D, X, params)

Regularized New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._mod.mod_reg(Y, D, X, params)

Regularized Method of Optimal Directions. The parameters are the same as for K-SVD.

### Incoherent Dictionary Learning

Dictionary Learning algorithms that jointly optimize the representation error and the coherence of the dictionary. The objective function consists in

$\min _{D, X}\|\mathbf{Y}-\mathbf{D} \mathbf{X}\|_{F}^{2}+\gamma\left\|\mathbf{D}^{T} \mathbf{D}-\mathbf{I}\right\|_{F}^{2}$

where $$\gamma > 0$$ is a trade-off parameter.

dictlearn.methods._ksvd.ksvd_coh(Y, D, X, params)

K-SVD algorithm with coherence reduction

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• D (ndarray of shape (n_features, n_components)) – The updated dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.aksvd_coh(Y, D, X, params)

Approximate K-SVD algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.uaksvd_coh(Y, D, X, params)

Updated-error Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.sgk_coh(Y, D, X, params)

Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.nsgk_coh(Y, D, X, params)

New Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

### Parallel Standard Dictionary Learning

Solves the Standard Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.pksvd(Y, D, X, params)

Parallel K-SVD algorithm

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• D (ndarray of shape (n_features, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.paksvd(Y, D, X, params)

Parallel Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.puaksvd(Y, D, X, params)

Parallel Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.psgk(Y, D, X, params)

Parallel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.pnsgk(Y, D, X, params)

Parallel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

### Parallel Regularized Dictionary Learning

Solves the Regularized Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.pksvd_reg(Y, D, X, params)

Regularized Parallel K-SVD

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• D (ndarray of shape (n_features, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.paksvd_reg(Y, D, X, params)

Parallel Regularized Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.puaksvd_reg(Y, D, X, params)

Parallel Regularized Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.psgk_reg(Y, D, X, params)

Parallel Regularized Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.pnsgk_reg(Y, D, X, params)

Parallel Regularized New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

### Parallel Incoherent Dictionary Learning

Solves the Incoherent Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.pksvd_coh(Y, D, X, params)

Parallel K-SVD with coherence reduction

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• D (ndarray of shape (n_features, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• D (ndarray of shape (n_features, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.paksvd_coh(Y, D, X, params)

Parallel Approximate K-SVD algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.puaksvd_coh(Y, D, X, params)

Parallel Updated-error Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.psgk_coh(Y, D, X, params)

Parallel Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.pnsgk_coh(Y, D, X, params)

Parallel New Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

### Kernel Dictionary Learning

In order to evade the linear character of the representation, a Kernel version of the Dictionary Learning problem is introduced. Through this method, the space of signals is extended to a nonlinear feature vector space. We associate with each signal $$\mathbf{y}$$ the feature vector $$\varphi(\mathbf{y})$$, where $$\varphi(\mathbf{y})$$ is a nonlinear function. The dictionary $$\mathbf{D}$$ is also extended to a nonlinear space by $$\varphi(\mathbf{y})\mathbf{A}$$, where $$\mathbf{A}$$ contains the coefficients of the dictionary. So, the DL problem is transformed into

\begin{split}\begin{align} \min _{D, \mathbf{X}} & \|\varphi(\mathbf{Y})-\varphi(\mathbf{Y})\mathbf{A} \mathbf{X}\|_{F}^{2} \\ \text { s.t. } & \left\|\mathbf{x}_{\ell}\right\|_{0} \leq s, \ell=1: N \\ & \left\|\mathbf{a}_{j}\right\|=1, j=1: n \end{align}\end{split}
dictlearn.methods._ksvd.ker_ksvd(K, A, X, params)

Kernel K-SVD

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_aksvd(K, A, X, params)

Kernel Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_uaksvd(K, A, X, params)

Kernel Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_sgk(K, A, X, params)

Kernel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_nsgk(K, A, X, params)

Kernel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._mod.ker_mod(K, A, X, params)

Kernel Method of Optimal Directions. The parameters are the same as for K-SVD.

### Regularized Kernel Dictionary Learning

A regularization version for the Kernel Dictionary Learning. The optimization problem consists in

$\min _{A, \mathbf{X}} \|\varphi(\mathbf{Y})-\varphi(\mathbf{Y}) \mathbf{A} \mathbf{X}\|_{F}^{2}+\mu\|\mathbf{X}\|_{F}^{2}$

where $$\mu > 0$$ is the regularization parameter.

dictlearn.methods._ksvd.ker_ksvd_reg(K, A, X, params)

Regularized Kernel K-SVD

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_aksvd_reg(K, A, X, params)

Regularized Kernel Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_uaksvd_reg(K, A, X, params)

Regularized Kernel Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_sgk_reg(K, A, X, params)

Regularized Kernel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_nsgk_reg(K, A, X, params)

Regularized New Kernel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

### Incoherent Kernel Dictionary Learning

A incoherent version for the Kernel Dictionary Learning. The optimization problem consists

$\min _{A, X}\|\varphi(\mathbf{Y})-\varphi(\mathbf{Y}) \mathbf{A} \mathbf{X}\|_{F}^{2}+\gamma\left\|\mathbf{A}^{T} \varphi(\mathbf{Y})^{T} \varphi(\mathbf{Y}) \mathbf{A}-\mathbf{I}\right\|_{F}^{2}$

where $$\gamma > 0$$ is a trade-off parameter.

dictlearn.methods._ksvd.ker_ksvd_coh(K, A, X, params)

Kernel K-SVD with coherence reduction

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_aksvd_coh(K, A, X, params)

Regularized Kernel Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_uaksvd_coh(K, A, X, params)

Kernel Updated-error Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_sgk_coh(K, A, X, params)

Kernel Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_nsgk_coh(K, A, X, params)

Kernel New Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

### Parallel Kernel Dictionary Learning

Solves the Kernel Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.ker_pksvd(K, A, X, params)

Parallel Kernel K-SVD

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_paksvd(K, A, X, params)

Parallel Kernel Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_puaksvd(K, A, X, params)

Parallel Kernel Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_psgk(K, A, X, params)

Parallel Kernel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_pnsgk(K, A, X, params)

Parallel Kernel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

### Parallel Regularized Kernel Dictionary Learning

Solves the Regularized Kernel Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.ker_pksvd_reg(K, A, X, params)

Parallel Regularized Kernel K-SVD

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_paksvd_reg(K, A, X, params)

Parallel Regularized Kernel approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_puaksvd_reg(K, A, X, params)

Parallel Kernel Regularized Updated-error Approximate K-SVD. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_psgk_reg(K, A, X, params)

Parallel Regularized Kernel Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_pnsgk_reg(K, A, X, params)

Parallel Regularized Kernel New Sequential Generalization of the K-means algorithm. The parameters are the same as for K-SVD.

### Parallel Incoherent Kernel Dictionary Learning

Solves the Incoherent Kernel Dictionary Learning problem by using a parallel perspective on the optimization problem. Instead of updating the atoms one by one, the atoms will be changed in parallel.

dictlearn.methods._ksvd.ker_pksvd_coh(K, A, X, params)

Parallel Kernel K-SVD with coherence reduction

Parameters
• Y (ndarray of shape (n_features, n_samples)) – Data matrix.

• A (ndarray of shape (n_samples, n_components)) – Initial dictionary, with normalized columns.

• X (ndarray of shape (n_components, n_samples)) – The sparse codes.

• params (dict) – Dictionary instance with provided parameters for the DL problem.

Returns

• A (ndarray of shape (n_samples, n_components)) – The update dictionary matrix.

• X (ndarray of shape (n_components, n_samples)) – The updated sparse codes.

dictlearn.methods._aksvd.ker_paksvd_coh(K, A, X, params)

Parallel Kernel Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._uaksvd.ker_puaksvd_coh(K, A, X, params)

Parallel Kernel Updated-error Approximate K-SVD with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._sgk.ker_psgk_coh(K, A, X, params)

Parallel Kernel Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.

dictlearn.methods._nsgk.ker_pnsgk_coh(K, A, X, params)

Parallel Kernel New Sequential Generalization of the K-means algorithm with coherence reduction. The parameters are the same as for K-SVD.