Kernel density matrices provide a simpler yet effective mechanism for representing joint probability distributions of both continuous and discrete random variables.  This abstraction allows the construction of differentiable models for density estimation, inference, and sampling, and enables their integration into end-to-end deep neural models.

MixLoRA is a type of PEFT method which construct a resource-efficient sparse MoE model based on LoRA.

MixLoRA

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts

Kernel Density Matrices for Probabilistic Deep Learning

A method to determine the dataset size used to LoRA fine-tune a model based on its weights by extracting the singular values of each LoRA matrix and training layer-specific nearest-neighbor classifiers.

Source	Kernel Density Matrices for Probabilistic Deep Learning
Year
Data Source	Papers with Code archive (CC BY-SA)