Skip to content

When is the serializer used? Why for training only but not eval #790

Description

@silentstar425

I noticed the data processing stage is extremely slow in eval stage. It seems that its caused by not serializing the params.

In training stage msg_type is 'model_para', while in eval stage it's 'evaluate'. But model params are transmitted in both stages.

federatedscope/core/message.py line 106 - 120:

def transform_to_list(self, x):

      if isinstance(x, list) or isinstance(x, tuple):
  
          return [self.transform_to_list(each_x) for each_x in x]
  
      elif isinstance(x, dict):
  
          for key in x.keys():
  
              x[key] = self.transform_to_list(x[key])
  
          return x
  
      else:
  
          if hasattr(x, 'tolist'):
  
              if self.msg_type == 'model_para':
  
                  return self.param_serializer(x)
  
              else:
  
                  return x.tolist()
  
          else:
  
              return x

We can see params are serialized in training stage, but not in other messages. I'm curious why it's designed like that, I’d really appreciate it if you could help me with this.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions