Skip to content

TypeAnalysis: accept nusw as well as inbounds on GEP indices - #3084

Open
vchuravy wants to merge 1 commit into
mainfrom
vc/typeanalysis-gep-nusw
Open

TypeAnalysis: accept nusw as well as inbounds on GEP indices#3084
vchuravy wants to merge 1 commit into
mainfrom
vc/typeanalysis-gep-nusw

Conversation

@vchuravy

@vchuravy vchuravy commented Aug 5, 2026

Copy link
Copy Markdown
Member

visitGEPOperator only types a GEP's indices as Integer when the GEP is inbounds. flang never emits inbounds for array element addresses — it emits nusw nuw, from XArrayCoorOp lowering in flang's CodeGen.cpp — so a Fortran array index is never typed.

That is self-reinforcing. The index does not become Integer because the GEP is not inbounds and the base is not yet a known Pointer; and the base never becomes a Pointer because pointer propagation requires either inbounds or indices that are already integral. Both the index chain and the array come out completely untyped. Activity analysis then cannot prove the arithmetic inactive, and integer address computations reach visitBinaryOperator's unhandled case as cannot handle unknown binary operator.

The change

bool indexCannotWrap = gep.isInBounds();
#if LLVM_VERSION_MAJOR >= 19
indexCannotWrap |= gep.hasNoUnsignedSignedWrap();
#endif

nusw is the premise this rule actually needs: the index is added as a signed byte offset which does not wrap, so it is an offset rather than something that might itself be a pointer. inbounds additionally guarantees the result stays within the object, which this rule does not rely on.

The pointer propagation below is deliberately left keyed on isInBounds(). Once the indices are typed Integer, its existing allIntegral path enables propagation on the next fixpoint iteration, so the base pointer is still typed without weakening that stronger premise.

Tests

  • test/TypeAnalysis/gepnusw.ll pins the rule on hand-written IR, with an inbounds twin that must analyse identically.
  • test/Fortran/TypeAnalysis/gep_nusw.f90 shows the same shape is what flang actually produces: it compiles a four-line subroutine, asserts the emitted IR contains getelementptr nusw nuw, and runs print-type-analysis over that output.

Both fail without the change. Note the Fortran test needs -DENZYME_FORTRAN=ON and flang as CMAKE_Fortran_COMPILER; otherwise it is skipped rather than run. It required registering a new test/Fortran/TypeAnalysis subdirectory.

Effect

On a whole-program Fortran module differentiating DVODE, cannot handle unknown binary operator drops from 27 to 17 — every failure in dvindy and dvjust, all of them mul. The remaining 17 are add on statistics counters, an unrelated cause.

check-typeanalysis shows no newly failing tests, compared by test name against the same build with only this hunk reverted. Verified twice: against LLVM 22 on this branch, and against LLVM 24 on a branch carrying the in-flight LLVM API ports.

Draft because I would like a second opinion on whether relaxing only the index rule, and leaving pointer propagation on isInBounds(), is the division you want — the alternative is to relax both.

Not covered: flang also emits some GEPs with no no-wrap flags at all (getelementptr [8 x i8], ptr %0, i64 %9), which this does not help.

visitGEPOperator only typed a GEP's indices as Integer when the GEP was
`inbounds`. flang never emits `inbounds` for array element addresses -- it
emits `nusw nuw`, from XArrayCoorOp lowering in flang's CodeGen.cpp -- so a
Fortran array index was never typed.

That is self-reinforcing: the index is not Integer because the GEP is not
inbounds and the base is not yet a known Pointer, and the base never becomes a
Pointer because pointer propagation requires either inbounds or indices that
are already integral. Both the index chain and the array come out completely
untyped. Activity analysis then cannot prove the arithmetic inactive, and
integer address computations reach visitBinaryOperator's unhandled case as
"cannot handle unknown binary operator".

`nusw` is the premise the rule actually needs: the index is added as a signed
byte offset which does not wrap, so it is an offset rather than something that
might itself be a pointer. `inbounds` additionally guarantees the result stays
within the object, which this rule does not rely on. The pointer propagation
below is deliberately left keyed on isInBounds(); once the indices are typed
Integer, its existing allIntegral path enables propagation on the next fixpoint
iteration without weakening that stronger premise.

Two tests: TypeAnalysis/gepnusw.ll pins the rule on hand-written IR, with an
inbounds twin that must analyse identically, and Fortran/TypeAnalysis/gep_nusw.f90
shows the same shape is what flang actually produces. Both fail without the
change.

On a whole-program Fortran module differentiating DVODE this takes "cannot
handle unknown binary operator" from 27 to 17, eliminating every failure in
dvindy and dvjust; the remainder are a separate, unrelated cause. check-enzyme
and check-typeanalysis show no newly failing tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vchuravy
vchuravy force-pushed the vc/typeanalysis-gep-nusw branch from 8d12241 to bbdaa9c Compare August 5, 2026 14:42
@vchuravy
vchuravy requested review from joewallwork and wsmoses August 5, 2026 14:44
@vchuravy
vchuravy marked this pull request as ready for review August 5, 2026 14:45
// `inbounds`, for array element addresses.
bool indexCannotWrap = gep.isInBounds();
#if LLVM_VERSION_MAJOR >= 19
indexCannotWrap |= gep.hasNoUnsignedSignedWrap();

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wsmoses not sure if this is 100% legal, but flang loves to emit nusw

@joewallwork joewallwork added the fortran Related to Enzyme's Fortran bindings label Aug 10, 2026

@joewallwork joewallwork left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of this is beyond what I understand but I have a comment on the test including Fortran code.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a feeling that the Fortitude linter I propose to introduce in #3130 for Fortran source would complain here that we have a subroutine that isn't contained in a program or module. Would this type of test still work if the subroutine were put inside a program or module?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fortran Related to Enzyme's Fortran bindings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants