NMF

non-negative matrix factorization, NMF

Fundamental Concept

• NMF is a dimensionality reduction method in which features and labels are both positive.
• Easy to model when dealing with image data, etc
• It is an algorithm that is applied only when all the components of the original matrix are positive.
  • The original matrix element must be positive.
  • After matrix decomposition, the matrix element is positive.
  • In NMF, there is no restriction that each dimension is orthogonal.
• PCA always uses orthogonal eigenvectors, making it difficult to find associations between data, but NMF can grasp the nature of the data.

Algorithm

• NMF do dimensionality reduction by decompose original data to 2 matrix

• Calculation Process
  1. Initialize W, H as a positive value
  2. Treat H as a constant and renew W
  3. Treat W as a constant and renew H
  4. End calculation when W, H converges to the original matrix

Sample Code

from sklearn.decomposition import NMF
from sklearn.datasets import make_blobs

centers = [[5, 10, 5], [10, 4, 10], [6, 8, 8]]

X, _ = make_blobs(centers=centers)

n_components = 2

model = NMF(n_components=n_components)

model.fit(X)
W = model.transform(X)
H = model.components_

print(W)
print(H)

[[1.19766994 0.13358679]
 [0.66746879 0.99088947]
 [0.6042335  1.01905346]
 [0.86681641 0.82490118]
 [1.48259625 0.08189441]
 [0.88876181 0.70164654]
 [0.69470766 0.79088102]
 [0.91115847 0.96651117]
 [0.80758121 0.77306129]
 [0.69905185 0.97413211]
 [0.85577822 0.57333738]
 [0.54641323 0.95454392]
 [0.73592249 1.09260989]
 [0.84008852 0.60879354]
 [0.88219332 0.77463386]
 [0.57804313 0.9914767 ]
 [1.02935213 0.65261928]
 [1.4412191  0.10039533]
 [1.49620828 0.17994642]
 [0.55370405 1.09346932]
 [0.78615864 0.83027296]
 [0.73595104 1.13939672]
 [0.509426   1.01769641]
 [1.42934294 0.06149644]
 [0.65137888 0.98170796]
 [1.36629018 0.14135849]
 [0.6038439  1.02659722]
 [1.19968199 0.08639518]
 [0.66073495 1.09830897]
 [0.88855344 0.89465474]
 [0.75247428 0.93819426]
 [0.73169429 0.84280388]
 [0.6598703  0.96468009]
 [1.32375515 0.        ]
 [0.61609098 0.91478167]
 [0.91447927 0.77497417]
 [0.88332728 0.64917029]
 [0.92159929 0.52058038]
 [1.29171335 0.12319609]
 [0.56227751 1.09897048]
 [1.39814087 0.18540118]
 [0.90242298 0.65935956]
 [0.52724319 1.12649579]
 [0.82819233 1.04809157]
 [0.60686853 1.02292252]
 [0.67583459 0.7702366 ]
 [1.44746301 0.06413168]
 [1.36425004 0.20016573]
 [0.89278407 0.75298045]
 [0.76914718 0.88557797]
 [0.88875379 0.64780441]
 [0.78422971 0.95356309]
 [1.35364677 0.05120759]
 [1.3318964  0.16184274]
 [1.40546062 0.16930998]
 [0.85360164 0.69411615]
 [1.33266278 0.16265007]
 [1.04182543 0.85077411]
 [1.34349061 0.15812565]
 [0.81406299 0.78620922]
 [0.8916262  0.81945665]
 [0.95880846 0.73921369]
 [0.88251054 0.69367952]
 [1.48188345 0.09134134]
 [1.04513583 0.84862865]
 [0.44286613 1.2816661 ]
 [0.52136123 0.98925529]
 [1.3262648  0.0018028 ]
 [1.33656138 0.06243606]
 [1.1959514  0.06763811]
 [1.3083053  0.14412448]
 [0.56065484 0.99295874]
 [1.07917815 0.45525339]
 [0.83121797 0.82285167]
 [1.41634831 0.09679492]
 [0.59182344 1.09634675]
 [0.63458217 0.90249339]
 [1.42602168 0.26449149]
 [0.97138147 0.73092181]
 [1.34621791 0.0605946 ]
 [1.37614465 0.12212213]
 [1.01944936 0.67597182]
 [0.75410776 1.04493802]
 [1.330216   0.36247901]
 [1.127936   0.23453544]
 [0.8022906  0.90076573]
 [1.33349605 0.12334863]
 [0.95921947 0.46886348]
 [0.92050375 0.65881885]
 [0.65141056 1.05754018]
 [0.49134421 1.0099088 ]
 [1.20535891 0.26343988]
 [1.18918774 0.25513411]
 [1.26670139 0.23837451]
 [0.5048205  1.10582409]
 [0.46833637 1.02242139]
 [1.16792676 0.08370171]
 [0.5342194  1.06576008]
 [0.55246285 1.06322998]
 [1.01809602 0.8103749 ]]
[[7.34113    2.09204933 7.53492881]
 [0.         8.52310408 0.73945398]]


C:\Users\bl4an\anaconda3\lib\site-packages\sklearn\decomposition\_nmf.py:289: FutureWarning: The 'init' value, when 'init=None' and n_components is less than n_samples and n_features, will be changed from 'nndsvd' to 'nndsvda' in 1.1 (renaming of 0.26).
  warnings.warn(

참고문헌

• 秋庭伸也 et al. 머신러닝 도감 : 그림으로 공부하는 머신러닝 알고리즘 17 / 아키바 신야, 스기야마 아세이, 데라다 마나부 [공] 지음 ; 이중민 옮김, 2019.