Skip to content

Only compile IsotropicModel.blocks - #11

Draft
Michal-Novomestsky wants to merge 1 commit into
PolymathicAI:mainfrom
Michal-Novomestsky:enable-torch-compile
Draft

Only compile IsotropicModel.blocks#11
Michal-Novomestsky wants to merge 1 commit into
PolymathicAI:mainfrom
Michal-Novomestsky:enable-torch-compile

Conversation

@Michal-Novomestsky

Copy link
Copy Markdown

Currently, IsotropicModel doesn't support torch.compile(). The issue seems to stem from bcs.tolist() which breaks the compiler:

DataDependentOutputException: aten._local_scalar_dense.default
While executing %item : call_method[target=item](args = (%getitem_11,))
Original traceback:
  File "src/model/walrus.py", line 301, in forward
    self.model(x, field_indices, bcs.tolist(), metadata=metadata)

Since most of the FLOPs live in self.blocks anyway (afaik), we can get most of the gains by only compiling them instead.

Note that this does not override torch.compile(model) (which will still break), only model.compile().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant