TypeAnalysis: accept nusw as well as inbounds on GEP indices - #3084
Open
vchuravy wants to merge 1 commit into
Open
TypeAnalysis: accept nusw as well as inbounds on GEP indices#3084vchuravy wants to merge 1 commit into
nusw as well as inbounds on GEP indices#3084vchuravy wants to merge 1 commit into
Conversation
visitGEPOperator only typed a GEP's indices as Integer when the GEP was `inbounds`. flang never emits `inbounds` for array element addresses -- it emits `nusw nuw`, from XArrayCoorOp lowering in flang's CodeGen.cpp -- so a Fortran array index was never typed. That is self-reinforcing: the index is not Integer because the GEP is not inbounds and the base is not yet a known Pointer, and the base never becomes a Pointer because pointer propagation requires either inbounds or indices that are already integral. Both the index chain and the array come out completely untyped. Activity analysis then cannot prove the arithmetic inactive, and integer address computations reach visitBinaryOperator's unhandled case as "cannot handle unknown binary operator". `nusw` is the premise the rule actually needs: the index is added as a signed byte offset which does not wrap, so it is an offset rather than something that might itself be a pointer. `inbounds` additionally guarantees the result stays within the object, which this rule does not rely on. The pointer propagation below is deliberately left keyed on isInBounds(); once the indices are typed Integer, its existing allIntegral path enables propagation on the next fixpoint iteration without weakening that stronger premise. Two tests: TypeAnalysis/gepnusw.ll pins the rule on hand-written IR, with an inbounds twin that must analyse identically, and Fortran/TypeAnalysis/gep_nusw.f90 shows the same shape is what flang actually produces. Both fail without the change. On a whole-program Fortran module differentiating DVODE this takes "cannot handle unknown binary operator" from 27 to 17, eliminating every failure in dvindy and dvjust; the remainder are a separate, unrelated cause. check-enzyme and check-typeanalysis show no newly failing tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
vchuravy
force-pushed
the
vc/typeanalysis-gep-nusw
branch
from
August 5, 2026 14:42
8d12241 to
bbdaa9c
Compare
vchuravy
marked this pull request as ready for review
August 5, 2026 14:45
vchuravy
commented
Aug 5, 2026
| // `inbounds`, for array element addresses. | ||
| bool indexCannotWrap = gep.isInBounds(); | ||
| #if LLVM_VERSION_MAJOR >= 19 | ||
| indexCannotWrap |= gep.hasNoUnsignedSignedWrap(); |
Member
Author
There was a problem hiding this comment.
@wsmoses not sure if this is 100% legal, but flang loves to emit nusw
joewallwork
reviewed
Aug 11, 2026
joewallwork
left a comment
Collaborator
There was a problem hiding this comment.
Most of this is beyond what I understand but I have a comment on the test including Fortran code.
Collaborator
There was a problem hiding this comment.
I have a feeling that the Fortitude linter I propose to introduce in #3130 for Fortran source would complain here that we have a subroutine that isn't contained in a program or module. Would this type of test still work if the subroutine were put inside a program or module?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
visitGEPOperatoronly types a GEP's indices asIntegerwhen the GEP isinbounds. flang never emitsinboundsfor array element addresses — it emitsnusw nuw, fromXArrayCoorOplowering in flang'sCodeGen.cpp— so a Fortran array index is never typed.That is self-reinforcing. The index does not become
Integerbecause the GEP is notinboundsand the base is not yet a knownPointer; and the base never becomes aPointerbecause pointer propagation requires eitherinboundsor indices that are already integral. Both the index chain and the array come out completely untyped. Activity analysis then cannot prove the arithmetic inactive, and integer address computations reachvisitBinaryOperator's unhandled case ascannot handle unknown binary operator.The change
nuswis the premise this rule actually needs: the index is added as a signed byte offset which does not wrap, so it is an offset rather than something that might itself be a pointer.inboundsadditionally guarantees the result stays within the object, which this rule does not rely on.The pointer propagation below is deliberately left keyed on
isInBounds(). Once the indices are typedInteger, its existingallIntegralpath enables propagation on the next fixpoint iteration, so the base pointer is still typed without weakening that stronger premise.Tests
test/TypeAnalysis/gepnusw.llpins the rule on hand-written IR, with aninboundstwin that must analyse identically.test/Fortran/TypeAnalysis/gep_nusw.f90shows the same shape is what flang actually produces: it compiles a four-line subroutine, asserts the emitted IR containsgetelementptr nusw nuw, and runsprint-type-analysisover that output.Both fail without the change. Note the Fortran test needs
-DENZYME_FORTRAN=ONand flang asCMAKE_Fortran_COMPILER; otherwise it is skipped rather than run. It required registering a newtest/Fortran/TypeAnalysissubdirectory.Effect
On a whole-program Fortran module differentiating DVODE,
cannot handle unknown binary operatordrops from 27 to 17 — every failure indvindyanddvjust, all of themmul. The remaining 17 areaddon statistics counters, an unrelated cause.check-typeanalysisshows no newly failing tests, compared by test name against the same build with only this hunk reverted. Verified twice: against LLVM 22 on this branch, and against LLVM 24 on a branch carrying the in-flight LLVM API ports.Draft because I would like a second opinion on whether relaxing only the index rule, and leaving pointer propagation on
isInBounds(), is the division you want — the alternative is to relax both.Not covered: flang also emits some GEPs with no no-wrap flags at all (
getelementptr [8 x i8], ptr %0, i64 %9), which this does not help.