Decouple batch size and number of negatives

[stes]

This PR adds a new argument, num_negatives, to both the Loader and CEBRA classes (torch and sklearn API). This allows to stabilize training behavior by providing additional negative examples to the InfoNCE loss independent of the batch size. We leveraged this logic for the models trained in DCL.

Behavior

If the num_negatives = None (the default), the previous behavior is obtained for backwards compatibility and Loader.batch_size negative examples are drawn. If a different value is set, then the number of negative examples in all loaders will be set to num_negatives instead of Loader.batch_size.

The goodness of fit computation was also adapted to use the num_negatives.

API modification

While implementing this functionality, I noticed an inconsistency between single_session and multi_session solvers. In the single session, we passed self.batch_size through the get_indices function, while in multi_session we use self.batch_size directly. The 2nd behavior makes more sense in the context of the general class design. I deprecated passing the num_samples parameter and adapted the samplers accordingly.

[stes]

@cla-bot check

[cla-bot]

Thanks for tagging me. I looked for a signed form under your signature again, and updated the status on this PR. If the check was successful, no further action is needed. If the check was unsuccessful, please see the instructions in my first comment.

[Copilot]

Pull Request Overview

This PR introduces the ability to decouple batch size from the number of negative samples in contrastive learning by adding a new num_negatives parameter to both the Loader and CEBRA classes. This allows for more stable training behavior by providing additional negative examples to the InfoNCE loss independent of the batch size.

• Adds num_negatives parameter to Loader base class and CEBRA class APIs
• Updates all loader implementations to use num_negatives instead of duplicating batch_size
• Modifies goodness of fit computation to use num_negatives instead of batch_size

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
cebra/data/base.py	Adds num_negatives field to base Loader class with validation and deprecates num_samples parameter
cebra/data/single_session.py	Updates single session loaders to use num_negatives and removes num_samples parameter
cebra/data/multi_session.py	Updates multi-session loaders to use num_negatives and adds validation for unified loader
cebra/data/multiobjective.py	Updates multiobjective loaders to use num_negatives instead of batch_size
cebra/integrations/sklearn/cebra.py	Adds num_negatives parameter to CEBRA class and passes it to loaders
cebra/integrations/sklearn/metrics.py	Updates goodness of fit computation to use num_negatives_ instead of batch_size
tests/test_loader.py	Comprehensive test updates to validate num_negatives functionality across all loader types
tests/test_sklearn.py	Adds basic test for num_negatives parameter in CEBRA class
tests/test_sklearn_metrics.py	Updates goodness of fit tests to validate num_negatives behavior

[MMathisLab]

hey @stes is any of the copilot reviews useful? If not, we can merge, thanks for this!!

[stes]

@MMathisLab fine to merge from my end. Went through the copilot comments and made some further edits.