This Uses similar blocks as of adapters but changes the way adapter activation works by adding a novel Activation function to it . This allows ConGater block to manually control the activation of the gates which results in continuous controll of any desired attributes inside the model.

It is often desirable to distill the capabilities of large language models (LLMs) into smaller student models due to compute and memory constraints. One way to do this for classification tasks is via dataset synthesis. Prior approaches to synthesis use few-shot prompting, which relies on the LLM's parametric knowledge to generate usable examples. However, this leads to issues of repetition, bias towards popular entities, and stylistic differences from human text. We propose Synthesize by Retrieval and Refinement (SynthesizRR), which uses retrieval augmentation to introduce variety into the dataset synthesis process: as retrieved passages vary, the LLM is seeded with different content to generate its examples. We find that SynthesizRR greatly improves lexical and semantic diversity, similarity to human-written text, and distillation performance,

SynthesizRR

SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation

ConGater

Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapters

The time-causal and time-recursive scale-space representation is obtained by filtering any 1-D signal with the time-causal limit kernel, and provides a way to define a multi-scale analysis for signals, for which the future cannot be accessed and additionally the computations should be strictly time-recursive, in order to not require any complementary memory of the past beyond the temporal scale-space representation itself.

Source	Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapters
Year
Data Source	Papers with Code archive (CC BY-SA)