Different configurations for SAE

[edircoli]

Right now GEA relies on the Vanilla version of SAEs (MSE + L1 regularization). Other architectures, such as Top-k SAEs, could be implemented.

[edircoli]

To try next:

• TopK SAE: no L1 sparsity term, only keeping the top-K features and setting everything else to zero.
• ReLU SAE: SAE with ReLU activation at encoder output layer.
• Matryoshka SAE: nested activated features, giving more interpretability on hierarchical terms.