Ridge Sketch API

Copyright (c) Facebook, Inc. and its affiliates. All rights reserved.

This source code is licensed under the license found in the LICENSE file in the root directory of this source tree.

class ridge_sketch.RidgeSketch(alpha=1.0, fit_intercept=True, tol=0.001, max_iter=2000, sketch_size=None, solver='auto', random_state=0, operator_mode=False, algo_mode='auto', step_size=None, mom_beta=None, mom_eta=None, use_heuristic=False, accel_mu=0.0, accel_nu=0.0, verbose=0)

Ridge Regression class for implementing Ridge Sketch.

Parameters

  • alpha (float)) – regularization strength. Must be greater than 0. Default=1.0

  • fit_intercept (bool) – whether to use an intercept in model. Default = True

  • tol (float) – error tolerance for solution. The solver stops when the relative residual norm becomes smaller than the tolerance. Default = 1e-3

  • sketch_size (int) – defaults to min(sqrt(m), sqrt(n))

  • solver (str) – defaults to ‘auto’. If dimensions small, uses direct solver. Otherwise, randomized solver. Options are “direct” or “ridgesketch” or “conjugate gradients”.

  • random_state (int) – seed to use for pseudo random number generation.

  • lambda (float) – regularization parameter

  • max_iter (int) – maximum number of iterations

  • operator_mode (bool) – default to True. Defines the covariance matrix as an operator (true) or as numpy matrix (false)

  • algo_mode (str) – defaults to ‘auto’ uses classical Ridge Sketch method. If set to ‘mom’, it uses the momentum version. Default is increasing momentum when neither ‘mom_eta’,’step_size’ or ‘mom_beta’ is set. If set to ‘accel’, it uses the accelerated version.

  • step_size (float) – step_size parameter for momentum version. Default = 1.

  • mom_beta (float) – momentum parameter between 0 and 1. Default = .9.

  • mom_eta (float) – parameter between 0 and 1 (excluded). Default = .99 when “algo_mode” is set to ‘mom’, in this case the algorithm uses the decreasing ‘step_size’ and increasing ‘mom_beta’ attributes.

  • use_heuristic (bool) – use the heuristic ruling increasing momentum If set to True, the code runs the momentum version with unitary step size and increasing momentum parameter beta, which is capped at 0.5. Default = False.

  • mu (float) – defaults to zero. The first acceleration parameter.

  • nu (float) – defaults to zero. The first acceleration parameter.

coef_

coefficients containing weight vectors

Type

np.array

intercept_

0.0, if fit_intercept is False

Type

float

direct_solver(A, b)

Solves linear system and computes residual

fit(X, y)

Fit ridge regression model based on solver and data

Sets coef_ and intercept_ attributes by solving

min ||y - Xw||^2 + alpha * ||w||^2

Parameters

  • X (2D numpy.ndarray) – data matrix (m, m)

  • y (2D numpy.ndarray) – labels (m, 1)

predict(X)

Generates prediction based on inputs

set_momentum(mom_beta, mom_eta, use_heuristic)

Setting momentum parameters depending on the inputs

set_sketch_size(X)

Sets sketching dimension based on data

set_step_size(step_size, mom_beta)

Setting step size parameter depending on the algo mode

sketch_solver(A, b)

Solver using a subsample sketching matrix

sketch_solver_setup(X, y)

Chooses between primal or dual formulation Solves resulting linear system Ax=b using a sketch-and-project method

solve_system(A, b)

Solves linear system Ax=b, where A is a sparse or dense matrix