A method to determine the dataset size used to LoRA fine-tune a model based on its weights by extracting the singular values of each LoRA matrix and training layer-specific nearest-neighbor classifiers.

Kernel density matrices provide a simpler yet effective mechanism for representing joint probability distributions of both continuous and discrete random variables.  This abstraction allows the construction of differentiable models for density estimation, inference, and sampling, and enables their integration into end-to-end deep neural models.

Kernel Density Matrices for Probabilistic Deep Learning

DSiRe

Dataset Size Recovery from LoRA Weights

In our proposed concept, we use an Inception layer after a convolutional layer with a kernel size of 1, and then, after performing batch normalization and the SILU activation function, we concatenate the output of this layer with the input of the bottleneck and pass it to the next layer. Moreover, in the other branch of C3-AMP, after performing a convolutional layer, a max pooling layer is employed, and then after concatenating the output of this layer with the output of bottleneck series, another convolutional layer with a kernel size of 1 is utilized.

Source	Dataset Size Recovery from LoRA Weights
Year
Data Source	Papers with Code archive (CC BY-SA)