-
Notifications
You must be signed in to change notification settings - Fork 55
Open
Description
According to the current rules, a submitter may intentionally or unintentionally introduce sparsity in the results table.
For example, if they choose a different system name for each workload they submit on essentially the same system:
awesome_system_for_resnet50
awesome_system_for_retinanet
...
the results table will contain one row per workload. One issue is that such a submitter may get an unfair advantage when their system is picked for audit - with only one workload being subject to audit.
We should mould the rules to encourage results density by carefully defining what constitutes "essentially the same system" and what does not.
arjunsuresh
Metadata
Metadata
Assignees
Labels
No labels